A little more than a year ago, I posted a Ruby reading list consisting of blog posts I often recommend to our students at RubyLearning as well as some interesting articles. It’s time for the updated version (some links are the same as last time, some are new, some are gone):

• The Unofficial Ruby Usage Guide (tips, tricks and style guidelines)


• The Ruby Object Model – Structure and Semantics


• Seeing Metaclasses Clearly


• The Ruby Singleton Class


• A Ruby Metaprogramming Introduction


• The Fully Upturned Bin (memory management and garbage collection)


• Writing Ruby extensions


• A dozen (or so) ways to start sub-processes in Ruby: Part 1 (Part 2, Part 3)


• Understanding Ruby Blocks, Procs and Lambdas


• Class and Instance Variables In Ruby (also class instance variables)


• Untangling Evented Code with Ruby Fibers


• Tailin’ Ruby


• Demistifying IRB’s Commands


• Ruby Enumerable Magic: The Basics

Disclaimer: The fine folks at No Starch Press were nice enough to provide me with a review copy of this book, but this has not influenced this review in any way.

Where to start? The Metasploit Framework (MSF) is a very popular penetration testing tool used by security professionals the world over. It was previously written in Perl but underwent a complete rewrite for version 3, where the developers switched the project to Ruby. The tool unifies the various stages of penetration testing in convenient interfaces (“msfconsole” for interactive use and “msfcli” for scripting purposes): information gathering and storage, exploit and payload configuration, IDS and antivirus evasion and actually exploiting the system.

From this you probably can gather that Metasploit is quite big and complex, as well as in a state of constant flux. This makes it rather hard to write a definitive book on it, which is illustrated by the fact that shortly after this volume got published, the Metasploit team released version 4 of the framework. Considering these difficulties, I’m tempted to say that the authors have done a tremendous job describing MSF as it was at the time of writing.

Now for the actual content: after a foreword by Metasploit’s main developer HD Moore, there’s a little introduction section on penetration testing and the history of the framework. This is followed by the first chapter, which covers some penetration testing basics. After this the authors give a first introduction to the MSF, before dedicating a chapter each to various phases of pen testing, namely information gathering, vulnerability scanning and the actual exploitation. After this you’ll find a whole chapter on Meterpreter, covering various aspects of post-exploitation techniques. Once you get to this point, you should have a good idea about how Metasploit works in principle and how capable it is. The authors don’t stop here though, but use the following chapters to try to teach you about avoiding detection, client-side exploits and Metasploit’s auxiliary modules. By this point in the book it felt like I already had learned a lot, but then I realized that I’m only halfway through the book! There still were chapters on various topics, including the social-engineering toolkit which is built in the MSF and wireless exploitation with Karmetasploit. As a Ruby developer/dev ops guy I was really interested in the next couple of chapters, which deal with building your own modules and exploits as well as porting existing exploits to Metasploit and Meterpreter scripting. Wow, the authors definitely covered a lot of ground until here, but we are still not done, since there’s on more chapter on how to simulate your pen tests.

While the above shows what the book covered, it doesn’t say much on how it was covered. In my opinion the authors did a very good job, the text is easy to follow and to the point and helped by screenshots and transcripts of “msfconsole” sessions. Sure, most of this material is also available on the Metasploit Unleashed web site, but I like having it all in the form of one compact book. I noticed 2-3 places where the textual description and the content of the screenshot/transcript didn’t exactly match, which can lead to brief moments of confusion, but nothing dramatic.

If you are new to Metasploit and want to get up to speed quickly, it’s hard to imagine that you’ll find a better book at the moment. More experienced users of the framework should flip through it in a book store to decide how much they’ll really get from it, but it’s probably still a good book to have around, even if it’s just for the cheat sheet in Appendix B.

• Saddam’s demon seed
  Latif Yahia was forced to work as Uday Hussein’s (Saddam’s son) body double, now a movie has been made about his life.
• Indonesia: Military Documents Reveal Unlawful Spying in Papua
  Lately I’ve been quite interested in Indonesia, not least because it’s a place people don’t tend to talk about too much, which is kinda odd considering it’s the 4th biggest country in the world.
• How GitHub Works: Hours are Bullshit
  A 3 part series of blog posts explaining how GitHub works. This is not about technical stuff, but about working hours, internal communications and so on (part 2, part 3).
• For first time, more corn used for ethanol than livestock
  While food prices rise and people are starving, more corn is used for ethanol than for feeding animals (and we are not even talking feeding people here).
• I’m a phony. Are you?
  I too noticed this lack of self-confidence in a lot of IT people and occasionally myself. Do you have any experience with it?
• Don’t look down
  I’m always surprised by how many people adhere to the “as long as someone is worth of than me” principle.
• Why most people don’t finish video games
  I barely play video games anymore, but when I do I also prefer short and/or multiplayer ones.
• A Stick Figure Guide to the Advanced Encryption Standard
  Awesome post explaining crypto to the non-techies out there. In related news, a new potential weakness in AES as found this week.
• Karl Marx Is Hot
  When US economists start quoting Marx, I start wondering if the global economy is really screwed?
• 13-Year-Old Makes Solar Power Breakthrough by Harnessing the Fibonacci Sequence
  A young boy looks at nature, sees a pattern and turns it into something useful. Here’s an essay

he wrote for the Young Naturalist Award. Or maybe this all was a hoax, at least according to one removed post on Blogger.

After What the Hack in 2005 and the Chaos Communication Camp 2007 this was my third hacker camp and once again it was an inspiring, funny, chaotic and productive week of talks, hacking, partying, networking and more.

Monday was mostly spent driving to the camp, which all in all went rather smoothly. All hail to Tesco where we picked up a lot of equipment, including the super awesome tent we slept in. Tuesday was a good opportunity to get a first impression of the camp, see who’s there, starting to socialize and having a party.

On Wednesday the talks started and I saw three of them. Strong encryption of credit card information was really quite interesting and once again I was shocked by how insecure some very important aspects of our lives are. Strahlung im Weltall dealt with space radiation and was part of the Hackers in Space track of the conference. When you think web development is difficult, how about dealing with radiation that might randomly flip a bit? Last but not least GPRS Intercept was another rather shocking talk, showing that GPRS security is as bad as GSM’s.

Thursday I started with Die psychologischen Grundlagen des Social Engineerings, which was a really good and interesting talk by Stefan Schumacher, who’s a really nice guy who spent quite a bit of time at our village. Steal Everything, Kill Everyone, Cause Total Financial Ruin! was another social engineering related talk and very entertaining. I got to talk to Jayson later that night and he really is as adorable as he says ;-) One of the talks I was most looking forward to was Hacking DNA and I wasn’t disappointed. As an added bonus Marc took some time on Saturday to come over to our village and help us with some really useful information on how to get a biohacking group going at a hackerspace.

Friday I spent mostly socializing, but I also went to see 2 talks. She-Hackers: Millennials and Gender in European F/LOSS Subculture unfortunately was the worst lecture I saw at this camp, which is a real shame considering that the topic at hand was very interesting. In the evening I watched Certified programming with dependent types, which was a fairly interesting introduction to the Coq Proof Assistant. As often with Andreas’s talks the title and abstract suggested slightly more, but it still was a solid and interesting presentation.

Saturday started rather late for me due to Friday night’s party, but I still woke up in time for Moonbounce Radio Communication, held by members of Metalab’s amateur radio crew. Later that day the Metalab was on stage again, this time in the form of Rethinking online news, which was an awesome presentation that led to a nice little meetup of interested parties the next afternoon. Stay tuned on this front, there might be interesting things happening in the not too distant future! The most amazing part of the day for me though was taking a 20 minute flight in an Antonov An-2 from 1968, which was really cool and worth every cent of the 45 Euro it cost.

On Sunday I watched Data Mining Your City, which was not as interesting as I’d hoped for. It sadly also was the day where we had to say goodbye to the camp :-(

Random thoughts: It’s super inspiring to spend an entire week around several thousand intelligent and creative people and like always after such an event I feel invigorated and full of ideas. I also finally should learn not to count on getting any work on one of my projects done at the camp, it never really works. Sometimes I wonder why I even bother bringing my laptop, an iPad or Netbook would probably be sufficient. But then why would I sit in front of my computer working on the same stuff I’m working on for the rest of the year when I’m surrounded by loads of interesting things and people I haven’t met yet?

I <3 you, Chaos Communication Camp!

• Anarchists should be reported, advises Westminster anti-terror police
  Yeah, let’s all spy on our neighbors, that always worked out well…
• Not so gullible after all
  Interesting piece about psychological studies on how we react to advertising logos and slogans.
• How A 21-Year-Old Design Student’s Sleeping-Bag Coat Could Break The Cycle Of Homelessnes
  I love projects that are about empowering and really helping people in need.
• Geschlecht unbekannt
  A Canadian couple is keeping the gender of their third child a secret, because they “want to let the kid decide”. They also are quite unconventional in bringing up the 2 older siblings (in German).
• Executions Should Be Televised
  Wow, if you seriously start thinking about this, there are lots of ethical considerations to be made. I’ll try to get over my initial gut reaction and to ponder the arguments of both sides.
• The 40-year mystery of America’s greatest skyjacking
  The crazy story of the USA’s only unsolved skyjacking. How have I never heard about this before?
• Kenya’s ambitions for an outsourced future
  Kenya is trying to establish itself as an outsourcing destination to establish a second line of income next to tourism.
• 8 Reasons Young Americans Don’t Fight Back: How the US Crushed Youth Resistance
  Although this was originally written for the US, some if it applies all over the world.
• Felicia Day: Mogul In The Making
  A little look at the work of Felicia Day, author and main character of the web series The Guild.
• The Singularity is Far: A Neuroscientist’s View
  So the singularity might not be around the corner, what a surprise.

There won’t be an “Information Overload” next week as I’ll be attending the Chaos Communication Camp.

• Mac OS X 10.7 Lion: the Ars Technica review
  Every time a new version of OS X comes out, I’m really looking forward to the in-depth Ars Technica review, because they are always super detailed (the current one weighs in at 19 pages), critical and highly informative.
• Pinke Inderinnen
  Dalit (“untouchable”) untouchable women in a poor region of India of Indian formed a gang to fight for their rights (in German).
• Timeline of incidents
  The Geek Feminism wiki has a timeline of sexist incidents in geek communities. It’s sad we still need something like this in 2011.
• Researcher Makes a Career Developing Cocaine Vaccine, But Pharmaceutical Companies Won’t Produce It
  Let’s rather fight a “war on drugs”, hm? Somehow I do find the thought of an anti-addiction vaccine pretty scary though.
• Minority Rules: Scientists Discover Tipping Point for the Spread of Ideas
  Apparently if 10 percent of a population hold an unshakable belief, it will be adopted by the majority. I think that would explain a lot…
• The World’s Deadliest Distinction
  The world’s oldest people regularly seem to die when they reach 114.
• The Campaign to Terrify You About EMP
  I’m constantly surprised by what politicians will bring up. The comment thread is kinda interesting though, people talk about solar flares and electro-magnetic storms which would have the same effect as an EMP.
• Is This The Beginning Of A Global Revolution?
  A summary of the big political changes we saw in recent month. It’s a bit strange how this article conflates revolutions and civil uprisings with democratic elections, but it’s still a good roundup.
• The Secret Ingredient In Your Orange Juice
  While I always have a certain hope that food laws in the EU are stricter than in the US, some of this probably still applies to juices sold here.
• Neue Medien und alte Geister
  It’s sometimes reassuring to read blog posts by politicians like Peter Tauber (CDU) who do understand the internet and new media (in German).

This week Tom — who is now a happily married man :-) — and yours truly finally read another paper and since I’m sick and barely left bed for the last 2 days I decided to fight my boredom by writing about it.

The paper
“Multiple Aspect Ranking using the Good Grief Algorithm” by Benjamin Snyder and Regina Barzilay from the MIT CS and AI Lab.

• Paper
• Presentation
• Data and code

Summary
The paper is in the field of sentiment analysis — extracting opinions from a text — and uses restaurant reviews as a corpus. The prime assumption is that such texts will contain more than one opinion (e.g. quality of food, price range, interior, quality of service), which the authors believe is not properly reflected by previous work in this area, where one opinion per text is assumed.

The problem the authors try to solve is assigning a rank from a fixed scale (e.g. 1-5) to several related aspects. They dismiss the easy approach of treating every aspect as a separate ranking problem, since they believe that a real text relates the different aspects in a coherent way (e.g. through phrases like “but”, “one problem was” etc.).

They built on previous work in the natural language processing field, namely linear models trained with “Perceptron” and extended this framework with a sort of “meta-model” that predicts relations (agreement or disagreement) between the individually ranked aspects. By relating the different aspects in such a way, it’s easier to reflect the contrasting views in real text like “good BUT pricey” in a meaningful way.

What follows is some math, where each input (a review) gets assigned an m-dimensional ranking vector (reflecting m aspects). For example if you try to rank 3 different aspects (food, service, price) on a scale of 1-5 a ranking vector might look like <5,5,3> (“food and service were great, but it was a bit pricey”). The joint ranking model then combines the individual ranks with an “agreement model” to introduce “grief terms” which express dissatisfaction in a certain aspect. The algorithm than tries to minimize the sum of this “grief” — which is what gave the algorithm its name — in a joint rank. This step then gets incorporated into the training of the individual rankers (there’s some pseudo-code for the joint training), where features of an aspect get represented through presence or absence of words and word bigrams. This is obviously a very simplified description of the algorithm, but it seems moot to repeat the entire math here, which is very well laid out in the presentation. There’s also a very nice illustration about decoding and relating the various aspects at the end of the PDF.

What then follows is a comparison of the Good Grief algorithm with similar algorithms on a corpus of 4500 restaurant reviews ranking restaurants on 5 different aspects, where it outperforms the “competition” in a statistically relevant way.

Takeaway
As someone who works in an area where sentiment analyses could come in handy I do have an interest in the topic, but alas we don’t have the time and resources to develop our own system to do this properly. I once hacked something together with JRuby and the OpenNLP library, which wasn’t really sentiment analysis but an attempt to extract useful phrases from reviews. It was crude and had its faults, but worked surprisingly well for the amount of time it took to write. It did however get abandoned in a prototype stage and after reading the paper it’s quite clear to me that that probably was a good choice. Sentiment analysis is a non-trivial problem that can’t properly be tackled in the ad-hoc fashion we tried. Should this topic ever resurface at my company I’ll definitely try to go for a more elaborate and scientific approach.

Difficulty of the paper
I know that for some people the math might look scary, but if you take some time and look at it carefully you’ll notice that it’s actually rather simple. It probably doesn’t hurt to be acquainted with some basic concepts of NLP and machine learning, but if you have those it’s quite an enjoyable paper even if you are not an absolute crack in the field.

Further reading

• Natural language processing
• Sentiment analysis
• OpenNLP

• Why My Father Hated India
  A half Indian/half Pakistani sheds some light on the animosity between the two countries.
• How Khan Academy Is Changing the Rules of Education
  Very nice piece on the Khan Academy, including examples of how it’s used in class rooms as well as positive and negative feedback by educators.
• Chain World Videogame Was Supposed to be a Religion-Not a Holy War
  Wow, I completely missed this last year, but this is a super-interesting piece on an even more interesting indie game.
• You are not running out of time
  This really resonates with me and judging from the comments I’m not the only one.
• Marching with the SlutWalkers
  A Guardian story on the SlutWalk protest marches that are causing debate amongst feminists. Because it fits the topic, here’s a picture of a sticker I took in Vienna not so long ago (it roughly translates to “the power of definition means that only the aggrieved can decide if sexual borders were crossed”):

• Ex-Apple Designer Creates Teaching UI That "Kills Math" Using Data Viz
  I’m almost jealous all these great math teaching tools didn’t exist when I went to school. Ok, I am jealous.
• Why I talk openly about the deep dark hole I’m sitting in
  For various reasons this affects me on a very personal level and I can only agree with the message of this post.
• "In 30 Jahren wird es keinen Kapitalismus mehr geben"
  Will capitalism really disappear within the next 30 years? Sometimes it definitely feels like it.
• After almost 20 years, math problem falls
  Interesting result about approximating something NP-complete without something P-complete which might have a huge impact in several disciplines, from engineering to economics.
• Keepin’ It Cool: How the Air Conditioner Made Modern America
  I’m not a big fan of ACs, but this article isn’t only about them, but about how one piece of technology can change the world in profound ways.

In my current effort of trying to really learn and master Haskell, I spent the last two or so weeks reading the excellent Typeclassopedia by Brent Yorgey. Since I probably will have to go through it a couple more times, I took notes in Emacs org-mode and exported them to HTML. You can find the summary either on GitHub or here:

• GitHub repo
• Typeclassopedia on citizen428.net

• How Seawater Can Power the World
  Interesting NYT op-ed on the future of fusion energy.
• Zoobotics
  Robot researchers are modeling their creations after animals now, assuming that it’s hard to come up with a better design than evolution.
• Edward Tufte’s "Slopegraphs"
  I really like this under-used chart style, I think it conveys meaning very well.
• Groundhog Decade
  Very good piece on how Hollywood apparently is dead set on repeating the mistakes of the music industry.
• Why Programming Languages?
  A programming language researcher explains why there are so many different programming languages and why we’ll likely keep on seeing new ones.
• The Menace Within
  40 years after the Stanford Prison Experiment some key players tell about their experience and what they took away from the experiment.
• Misattribution of Arousal
  I love all those insights into how our brains work. This article deals with the misattribution of arousal (not only of the sexual kind) to other people.
• How Digital Detectives Deciphered Stuxnet, the Most Menacing Malware in History
  If you have the slightest interest in IT security, you’ll love this article, but even if you usually don’t enjoy this topic the story still reads like the plot of a Hollywood movie.
• Tower of David: 45 Floors of Favela Chic
  One of Latin America’s tallest buildings is now squatted by around 2500 people, many of them escaping live on the streets.
• My Summer at an Indian Call Center
  An American citizen works in an Indian call center for a summer.