Forrest Iandola

A few months ago, I blogged about how to reduce AlexNet image resolution without giving up much ImageNet accuracy. I’m not quite sure what I did wrong, but I’m not able to reproduce those results. I’m sorry for leading you down the wrong path, and thanks to the 10 people who wrote me with questions about this!

New numbers
Here’s a revised version with numbers that I’ve been able to produce consistently with ImageNet-1k on an NVIDIA K40, where all the accuracy deltas are compared to 256×256 AlexNet:

As you can see, the drop in accuracy for 128×128 AlexNet is larger than what I listed in my previous blog post. Oops.

After trying a few other DNN architectures, I identified an architecture that I’m calling VGG_F_extralayer [4]. With VGG_F_extralayer, we claw our way back up above 50% top-1 accuracy, while maintaining some speed benefits due to 128×128 images.

There are a few differences between VGG_F and VGG_F_extralayer:
1. VGG_F_extralayer has an additional 1×1 conv layer with 256 filters after conv4. (Going deeper sometimes improves accuracy.)
2. In its final pooling layer, VGG_F_extralayer does average pooling instead of max pooling. (In general, I often find that average-pooling near the end of a DNN provides a moderate bump in accuracy.)
3. The conv1 layer has 5×5 instead of 11×11 filters. (11×11 would probably give similar accuracy.)
4. The strides in VGG_F_extralayer for conv1 and pool1 are slightly different than VGG_F (see the details in the VGG_F_extralayer prototxt file [4]).

What’s next?
There are plenty of open questions, such as “which of these modifications have the biggest impact on accuracy?” I invite you to explore them.

[1] A. Krizhevsky, I. Sutskever, G.E. Hinton. ImageNet Classification with Deep Convolutional Neural Networks. NIPS, 2012.
[2] K. Chatfield, K. Simonyan, A. Vedaldi, A. Zisserman. Return of the Devil in the Details: Delving Deep into Convolutional Nets. BMVC, 2014 .
[3] VGG_F prototxt: https://gist.github.com/ksimonyan/a32c9063ec8e1118221a
[4] VGG_F_extraLayer prototxt: http://www.forrestiandola.com/blog/wp-content/uploads/2015/07/vgg_f_extraLayer_trainval.prototxt

Making scrambled eggs is as easy as “scramble some eggs and cook them,” right? Well, sort of. There are a lot of subtle choices!

Martha Stewart endorses the “scramble some eggs and cook them” philosophy.

Dani Spies adds water when beating the eggs, and she cooks them in coconut oil.

Gordon Ramsay adds a few more flourishes, taking the eggs on and off the heat. This, combined with continuous stirring of the eggs, produces more of a custard than traditional scrambled eggs.

My work with Profs. William D. Gropp and Wen-Mei Hwu and research scientist Mike Showerman is featured in the National Center for Supercomputing Applications (NCSA) Newsletter!

In March 2010, a team of fifteen University of Illinois and NCSA computer scientists including professors, graduate students, and undergrads set out push the limits of energy efficiency in cluster computing. Toward this goal, we collaborated with NVIDIA’s Sean Treichler to design a 128-node cluster that uses commodity hardware and NVIDIA graphics processing units (GPUs). Earlier this fall, we constructed and benchmarked the cluster for efficiency and performance.

The jury is still out as to where the our new 128-node cluster ranks on the Green500, which is a list of the world’s most efficient clusters and supercomputers. The Green500 rankings are slated to be announced at the the upcoming Supercomputing conference.

While perusing the blog of Chris Sells of Microsoft, I came across the best interview question I’ve ever seen:

I walked into my first technical interview at Microsoft, and before I could say anything, the woman says, You’re in an 8×8 stone corridor. I blink and sit down.

Interviewer: The prince of darkness appears before you.

Me: You mean, like, the devil?

Interviewer: Any prince of darkness will do.

Me: Ok.

Interviewer: What do you do?

Me: Can I run?

Interviewer: Do you want to run?

Me: Hmm I guess not Do I have a weapon?

Interviewer: What kind of weapon do you want?

Me: Um something with range?

Interviewer: Like what?

Me: Uh a crossbow?

Interviewer: What kind of ammo do you have?

Me: Ice arrows?

Interviewer: Why?

Me: Because the prince of darkness is a creature made of fire???

Interviewer: Fine so what do you do next?

Me: I shoot him?

Interviewer: No what do you do?

Me:

Interviewer: You WASTE him! You *WASTE* the prince of darkness!!

Me: Holy crap, what have I gotten myself into?

What’s the funniest interview question you have been asked? Reply in the comments!

Earlier this week, the University of Illinois Daily Illini published a database of salaries for all University of Illinois faculty and staff. Top salaries included football coach Ronald Zook at $1,052,500.10 and Engineering Dean and Donald Biggar Willett Professor of Engineering Ilesanmi Adesida at $309,466. The range of salaries also included Special Education Clerical Assistant Vikas K. Singh at $7,903.48 and Visiting Lecturer of Dance Denis Chiaramont at $6,468.84.

Although the Daily Illini article offered little text or opinion content, the article was quite inflammatory. One online commenter posted, “Understanding to the penny how much each and every person in your own office makes is a demoralizing and humiliating experience. It creates jealousy and hostility in an already trying work environment, and can sour even the best of friendships.” However, the Daily Illini did little more than to neatly organize publicly available data in to a searchable database. In fact, University of Illinois salaries have been available here since long before the Daily Illini created the database.

The Daily Illini salary database is just one of many instances where organizing publicly shared data sparks significant public attention. Last year, Caltech computer science graduate student Virgil Griffith looked up the most popular music on Facebook profiles of students at each United States university. Then, Griffith compared the 133 most popular music artists on his list with the average incoming SAT scores of the universities where the music was popular. Griffith compared music artists and SAT scores in a graph called Music That Makes You Dumb.

The graph suggests that students who like Beethoven are likely to have scored above 1350 on the SAT, and students who listen to Lil’ Wayne may have scored less than 900. While Griffith did little more than to organize freely available data, Music Makes You Dumb garnered attention from the Wall Street Journal, The Washington Post, and The New Yorker.

Beyond salaries, SAT scores, and music preferences, there are countless public online resources that could be organized and published as databases or graphs for increased public attention. For instance, driving records available for Champaign County, IL are available for free in an online database. While the database is a bit crude and inconvenient to use, it is still easy to find driving records for any of your friends in Champaign County.

For instance, on November 23, 2009, University of Illinois accounting professor Ira Solomon was issued a $75 ticket for driving his 2000 Volvo at 11-14mph above the speed limit at the corner of Lincoln and Stoughton–just a block from where I live–Urbana, IL. Luckily, with a Daily Illini-reported salary of $294,230.16, I doubt Prof. Solomon was particularly distraught over his $75 ticket.