NerdyRoom™

I confess, I use ms excel from time to time. I find it easy to shuffle around datasets and to create simple diagrams to see if my data matches my expectations. But as soon as it comes to really plotting the data for publishing I curse myself for never having learned to use a sophisticated command line tool and revert back to my crappy configured gnuplots. Normally I just copy the parts of the dataset I want from the various excel sheets into plain txt files to have gnuplot plot them. Damn MS uses \r instead of \n so gnuplot doesn’t plot these files. This is where tr comes to the rescue:

tr '\r' '\n' < input.dat > output.dat

Why do I always end up using these damn old *nix tools – OMG – I’m getting old . . .

I confess, I use ms excel from time to time. I find it easy to shuffle around datasets and to create simple diagrams to see if my data matches my expectations. But as soon as it comes to really plotting the data for publishing I curse myself for never having learned to use a sophisticated command line tool and revert back to my crappy configured gnuplots. Normally I just copy the parts of the dataset I want from the various excel sheets into plain txt files to have gnuplot plot them. Damn MS uses \r instead of \n so gnuplot doesn't plot these files. This is where tr comes to the rescue: tr '\r' '\n' < input.dat > output.dat Why do I always end up using these damn old *nix tools - OMG - I'm getting old . . .

Today, I had the “pleasure” to submit a patch to kernel.org (not the kernel list, but one of the InfiniBand lists there). Sadly, I found out, that they are really restrictive about the style of the mail, patch, etc, see the FAQ of the kernel list. But there is a nice tutorial to avoid most of the pitfalls.

If someone of you has the same task, then this how-to is a good starting point.

Today, I had the "pleasure" to submit a patch to kernel.org (not the kernel list, but one of the InfiniBand lists there). Sadly, I found out, that they are really restrictive about the style of the mail, patch, etc, see the FAQ of the kernel list. But there is a nice tutorial to avoid most of the pitfalls. If someone of you has the same task, then this how-to is a good starting point.

using git was kind of painful as you had to configure “apt” to keep track of the changes with a “Post-Invoke” command. Determining the caller of apt was something ugly like “caller=$(ps axww | grep “^ *$$” -B3 | grep ” apt-get ” | head -n1 | sed ‘s/^.*\(apt-get .*\)/\1/’ )”. Forgetting to “chmod og-rwx /etc/.git” made your password shadows world readable. No fun at all . . .

The simple solution to all of that is called: etckeeper

It uses git per default but can use mercurial(hg), bazaar, or darcs as well. Supported packet managers are apt(dpkg), yum(rpm), pacman-g2 and probably more. Etckeeper does a commit every day per default, but as its a normal git (or hg …) one can manually commit any time (by using git directly or through “etckeeper commit message”). Everything (else) can be configured at /etc/etckeeper – funny coincidence – changes there will be tracked by etckeeper as well. Sounds like a self fulfilling prophecy – doesn’t it

BTW: Normally, etckeeper is run inside /etc but with the command “etckeeper init [-d directory] one can keep a clone of /etc elsewhere (read: backup).

using git was kind of painful as you had to configure "apt" to keep track of the changes with a "Post-Invoke" command. Determining the caller of apt was something ugly like "caller=$(ps axww | grep "^ *$$" -B3 | grep " apt-get " | head -n1 | sed 's/^.*\(apt-get .*\)/\1/' )". Forgetting to "chmod og-rwx /etc/.git" made your password shadows world readable. No fun at all . . . The simple solution to all of that is called: etckeeper It uses git per default but can use mercurial(hg), bazaar, or darcs as well. Supported packet managers are apt(dpkg), yum(rpm), pacman-g2 and probably more. Etckeeper does a commit every day per default, but as its a normal git (or hg ...) one can manually commit any time (by using git directly or through "etckeeper commit message"). Everything (else) can be configured at /etc/etckeeper - funny coincidence - changes there will be tracked by etckeeper as well. Sounds like a self fulfilling prophecy - doesn't it ;) BTW: Normally, etckeeper is run inside /etc but with the command "etckeeper init [-d directory] one can keep a clone of /etc elsewhere (read: backup).

System:

• Debian testing – “wheezy”
• Gnome 3.0+6
• Network Manager 0.9.2.0-1

Issue:

• WLAN works fine with NetworkManager
• Wired Device (ethX) is marked as “unmanaged” and doesn’t work even if it gets DHCP-lease/IP-address
• /etc/network/interfaces looks correct

Solution:

Change /etc/NetworkManager/NetworkManager.conf :

...
[ifupdown]
managed=true
...

Reload Settings:

/etc/init.d/network-manager force-reload

System: Debian testing - "wheezy" Gnome 3.0+6 Network Manager 0.9.2.0-1 Issue: WLAN works fine with NetworkManager Wired Device (ethX) is marked as "unmanaged" and doesn't work even if it gets DHCP-lease/IP-address /etc/network/interfaces looks correct Solution: Change /etc/NetworkManager/NetworkManager.conf : ... [ifupdown] managed=true ... Reload Settings: /etc/init.d/network-manager force-reload

Yesterday, I decided to kick my Ubuntu LTS from my Thinkpad T500 and give Debian with Gnome 3 a chance. I tried Ubuntu with Unity in spring and was shocked. Besides all the little Bugs which are excusable I don’t like the way it feels. With the next LTS, I probably will give it a second chance because I like Ubuntu.

But what’s about Gnome 3? Really deep changes were made. GTK, that is used, was raised to the next major version and fundamental changes took place in the control concept. These are just two reasons why development is still in a kind of beta status. I think Debian is a better platform for Gnome 3 at the moment than Ubuntu, because Ubuntu comes with several pre-configurations and pre-installations which might mess up working with software not adapted to it. Usually the Debian guys don’t take it too seriously with their release cycles and prefer stability to actuality.

Core:
Debian testing “wheezy” ( dist-upgrade from stable )
Gnome 3.0+6

Installation and configuration went way less direly than expected. All looks new, nice, and fancy and performs very good at first sight. Most of the new features make sense to me and the period of settling in was just a few hours. I don’t understand why they have “loan” so much look and feel from Apples’ MacOS, which they “loan” from others, but if it makes someone happy I’m too.

Things I like:

• All hardware works out of the box – THX Debian
• It seams all functionality approached a few clicks
• The new management of multiple Desktops
• Arrangement of (sub)menus and icons
• Window fit-in by dragging it to the edges of the screen
• The new calendar
• General behaviour and usage of modifier keys
• Stability (no crashes to that point)
• Performance

Yesterday, I decided to kick my Ubuntu LTS from my Thinkpad T500 and give Debian with Gnome 3 a chance. I tried Ubuntu with Unity in spring and was shocked. Besides all the little Bugs which are excusable I don't like the way it feels. With the next LTS, I probably will give it a second chance because I like Ubuntu. But what's about Gnome 3? Really deep changes were made. GTK, that is used, was raised to the next major version and fundamental changes took place in the control concept. These are just two reasons why development is still in a kind of beta status. I think Debian is a better platform for Gnome 3 at the moment than Ubuntu, because Ubuntu comes with several pre-configurations and pre-installations which might mess up working with software not adapted to it. Usually the Debian guys don't take it too seriously with their release cycles and prefer stability to actuality. Core: Debian testing "wheezy" ( dist-upgrade from stable ) Gnome 3.0+6 Installation and configuration went way less direly than expected. All looks new, nice, and fancy and performs very good at first sight. Most of the new features make sense to me and the period of settling in was just a few hours. I don't understand why they have "loan" so much look and feel from Apples' MacOS, which they "loan" from others, but if it makes someone happy I'm too. Things I like: All hardware works out of the box - THX Debian ;) It seams all functionality approached a few clicks The new management of multiple Desktops Arrangement of (sub)menus and icons Window fit-in by dragging it to the edges of the screen The new calendar General behaviour and usage of modifier keys Stability (no crashes to that point) Performance Things I don't like (Bugs): Even more settings are hidden from the user than in Gnome 2 - common guys do you really think we're all stupid morons ?!? - Session saving and properties only reachable through console(WTF?); Window themes/styles not changaeble (or didn't find it by now) to have minimize and maximize buttons back... Fast command execution via Alt+F2 doesn't find any applications - use gnome-do All my applets are gone ;( - heard the will reappear later No automount of USB-Devices because the system thinks the are CDs...should be an Debian issue... I wanted to make a fancy screenshot for you but "Execution of 'gnome-screenshot' failed: Command not found" appeared when in "overview mode" My opinion is, if Gnome 3 matured a bit more and the next stable Debian will have it included, It could be most peoples desktop environment of choice.

Not so long ago we argued about programming languages and IDEs. As expected several worlds collide. So maybe there is no answer to the question what’s best, but I want to draw your attention to a nice IDE.

I needed a platform-independent and comfortable tool for programming. One would say “Use Eclipse” but my past with Java and Eclipse was bad. Or one would say “Use Emacs or Vim, stupid”, true but Linux tools on Windows mostly are pain in the ass. What I finally found was Code::Blocks, available for Linux, Windows and MAC. It’s a nice, fast and in my opinion mighty IDE for C and C++ (other languages via plugins). It needs a bit time to get familiar with it but there are very powerful features making this worthwhile. I don’t want to waste your time anny longer so try by your own.

Little hint: the version receivable from the last Ubuntu LTS is ooooold so download debs from the website.

URL: www.codeblocks.org

Not so long ago we argued about programming languages and IDEs. As expected several worlds collide. So maybe there is no answer to the question what's best, but I want to draw your attention to a nice IDE. I needed a platform-independent and comfortable tool for programming. One would say "Use Eclipse" but my past with Java and Eclipse was bad. Or one would say "Use Emacs or Vim, stupid", true but Linux tools on Windows mostly are pain in the ass. What I finally found was Code::Blocks, available for Linux, Windows and MAC. It's a nice, fast and in my opinion mighty IDE for C and C++ (other languages via plugins). It needs a bit time to get familiar with it but there are very powerful features making this worthwhile. I don't want to waste your time anny longer so try by your own. Little hint: the version receivable from the last Ubuntu LTS is ooooold so download debs from the website. URL: www.codeblocks.org

Nur um es mal irgendwo vermerkt zu haben: es scheint NAT für IPv6 zu geben.

http://www.heise.de/newsticker/meldung/Netfilter-Entwickler-arbeiten-an-NAT-fuer-ip6tables-1384952.html

Nur um es mal irgendwo vermerkt zu haben: es scheint NAT für IPv6 zu geben. http://www.heise.de/newsticker/meldung/Netfilter-Entwickler-arbeiten-an-NAT-fuer-ip6tables-1384952.html

Hab da grad ein Whitepaper in die Finger bekommen (von Cisco). Die behaupten darin dass ihr, mit 10GigE bestückter Cluster, so schnell ist wie ein gleichwertiger Cluster mit InfiniBand QDR (40Gigabit wie sie selbst zugeben). Ich ja immer übervorsichtig bei Whitepapers von Firmen die einem Sachen verkaufen wollen und dann sich auch noch mit der Konkurrenz vergleichen.

Naja wie dem auch sei, die nehmen unteranderem einen Benchmark der von der Autoindustrie zur Crashsimulation benutzt wird. Ich hab dann mal hinterher gegoogelt und siehe da die Ergebnisse sind öffentlich und scheinen auch zu stimmen. Nur ist mir ein Detail aufgefallen, laut topcrunch.org hatte der IB Cluster (Ergebnisse von 2010) genau “RAM per CPU: 2″, was jetzt viel Interpretation zuläßt. 2GB pro Core, pro CPU, oder 2 RAM-Rigel unbekannter Größe pro CPU. Dem gegenüber steht zumindest ein Cisco Rack mit 96GB pro Node, was 8GB pro Core entspricht.

Ich mutmaße mal dass die hohe/vergleichbare 10GigE Performance durch ein paar Tricks zustande gekommen sein kann, aber kann es natürlich nicht beweisen:

• Es wurde mehr RAM benutzt damit die Anwendung schnell läuft (ein Indiz dafür ist u.a. die höhere Performance auf einem einzigen Node, s. Figure 2)
• Die Kommunikationslast der Benchmarks ist sehr gering, bzw die Benchmarks wurden speziell nach dem Kriterium ausgesucht
• Die Benchmarks arbeiten nur mit sehr kleinen Nachrichten, wodurch die 4x höhere Peakperformance von IB nicht sichbar wird.

Aber lest selbst und bildet euch eine Meinung. Vielleicht siehts von euch ja jemand anders und kann mir glaubhaft erklären warum bei 2 angeblich identische Clustern, die Anwendungen auf dem “von Spezifikationen her langsameren” Interconnect schneller laufen sollen.

Hab da grad ein Whitepaper in die Finger bekommen (von Cisco). Die behaupten darin dass ihr, mit 10GigE bestückter Cluster, so schnell ist wie ein gleichwertiger Cluster mit InfiniBand QDR (40Gigabit wie sie selbst zugeben). Ich ja immer übervorsichtig bei Whitepapers von Firmen die einem Sachen verkaufen wollen und dann sich auch noch mit der Konkurrenz vergleichen. Naja wie dem auch sei, die nehmen unteranderem einen Benchmark der von der Autoindustrie zur Crashsimulation benutzt wird. Ich hab dann mal hinterher gegoogelt und siehe da die Ergebnisse sind öffentlich und scheinen auch zu stimmen. Nur ist mir ein Detail aufgefallen, laut topcrunch.org hatte der IB Cluster (Ergebnisse von 2010) genau "RAM per CPU: 2", was jetzt viel Interpretation zuläßt. 2GB pro Core, pro CPU, oder 2 RAM-Rigel unbekannter Größe pro CPU. Dem gegenüber steht zumindest ein Cisco Rack mit 96GB pro Node, was 8GB pro Core entspricht. Ich mutmaße mal dass die hohe/vergleichbare 10GigE Performance durch ein paar Tricks zustande gekommen sein kann, aber kann es natürlich nicht beweisen: Es wurde mehr RAM benutzt damit die Anwendung schnell läuft (ein Indiz dafür ist u.a. die höhere Performance auf einem einzigen Node, s. Figure 2) Die Kommunikationslast der Benchmarks ist sehr gering, bzw die Benchmarks wurden speziell nach dem Kriterium ausgesucht Die Benchmarks arbeiten nur mit sehr kleinen Nachrichten, wodurch die 4x höhere Peakperformance von IB nicht sichbar wird. Aber lest selbst und bildet euch eine Meinung. Vielleicht siehts von euch ja jemand anders und kann mir glaubhaft erklären warum bei 2 angeblich identische Clustern, die Anwendungen auf dem "von Spezifikationen her langsameren" Interconnect schneller laufen sollen.

I was quite surprised to see the speed up achieved by enabling TRIM support on Lion using TRIM Enabler. After I enabled it, I created a big file, deleted it (thus freeing/trimming the no longer used blocks) and rewriting another file of the same size:

jupp@x / > dd if=/dev/zero of=zero bs=1m count=$((6*1024))
6144+0 records in
6144+0 records out
6442450944 bytes transferred in 71.336086 secs (90311248 bytes/sec)
jupp@x / > rm zero                                                                                                                                                                                                                                                                                                                      jupp@x / > dd if=/dev/zero of=zero bs=1m count=$((6*1024))
6144+0 records in
6144+0 records out
6442450944 bytes transferred in 55.568795 secs (120149352 bytes/sec)

That’s a 1.33 speedup for sequential writing. This should not effect reading, of course. As you can imagine, the benefits for random IO on small blocks will be much higher than that.

This shows that the on-chip garbage collector in the SSD drive is (as expected) unable to do anything if full disk encryption is used, requiring the use of TRIM in order to keep write performance high. However, be warned: the use of TRIM could potentially compromise the invisibility of your main partition if using a fake partition for plausible deniability.

I’m using a MacbookPro6,2 and an Intel SSDSA2CW120G3.

Disclaimer: I’m aware that using dd in this fashion in order to achieve quantitive benchmarking results is not recommended. It was just a quick hack to demonstrate that it’s freakin faster with TRIM.

I was quite surprised to see the speed up achieved by enabling TRIM support on Lion using TRIM Enabler. After I enabled it, I created a big file, deleted it (thus freeing/trimming the no longer used blocks) and rewriting another file of the same size: jupp@x / &gt; dd if=/dev/zero of=zero bs=1m count=$((6*1024)) 6144+0 records in 6144+0 records out 6442450944 bytes transferred in 71.336086 secs (90311248 bytes/sec) jupp@x / &gt; rm zero jupp@x / &gt; dd if=/dev/zero of=zero bs=1m count=$((6*1024)) 6144+0 records in 6144+0 records out 6442450944 bytes transferred in 55.568795 secs (120149352 bytes/sec) That's a 1.33 speedup for sequential writing. This should not effect reading, of course. As you can imagine, the benefits for random IO on small blocks will be much higher than that. This shows that the on-chip garbage collector in the SSD drive is (as expected) unable to do anything if full disk encryption is used, requiring the use of TRIM in order to keep write performance high. However, be warned: the use of TRIM could potentially compromise the invisibility of your main partition if using a fake partition for plausible deniability. I'm using a MacbookPro6,2 and an Intel SSDSA2CW120G3. Disclaimer: I'm aware that using dd in this fashion in order to achieve quantitive benchmarking results is not recommended. It was just a quick hack to demonstrate that it's freakin faster with TRIM.

From http://www.esm.psu.edu/~ajm138/fortranexamples.html:

C programmers love to gloat that recursion can not be done in Fortran. When they say Fortran, they mean FORTRAN 77, since they absolutely refuse to acknowledge the existence of modern Fortran 90. (Fortran 90 came out over ten years ago. You would think they might know something about it by now!) But this example shows that even in FORTRAN 77 one can quickly and easily write routines that are recursive. So stick that in your complex variables, C programmers, and — oh wait, I forgot there are no complex variables in C. You have to manually define a complex data type. Not to mention having to write math functions like sin( ) and cos( ) to handle such data types.

Hahaaa … love it!

From http://www.esm.psu.edu/~ajm138/fortranexamples.html: C programmers love to gloat that recursion can not be done in Fortran. When they say Fortran, they mean FORTRAN 77, since they absolutely refuse to acknowledge the existence of modern Fortran 90. (Fortran 90 came out over ten years ago. You would think they might know something about it by now!) But this example shows that even in FORTRAN 77 one can quickly and easily write routines that are recursive. So stick that in your complex variables, C programmers, and -- oh wait, I forgot there are no complex variables in C. You have to manually define a complex data type. Not to mention having to write math functions like sin( ) and cos( ) to handle such data types. Hahaaa ... love it!