Lightroom & Photo Workflow

I seem to have settled into a reasonable work flow with my photography and Lightroom.

I have multiple memory cards for each camera. Two 2 gb cards for the Canon SD700 IS - shooting jpeg gives me about 700 photos per card or about 15 min. of video at the highest quality (640x480, 30fps). I can't remember the last time I filled up one of these cards. I have four 4 gb cards for the Pentax K10D - shooting DNG (raw) I get about 240 photos per card and I do fill these up sometimes. For the Pentax I always carry at least one spare memory card and a spare battery.

I have spare batteries for both cameras but seldom need them. The last a long time and most of the time it's easy to recharge them. The exception is in more remote locations where power isn't readily available.

When traveling I usually download/import once a day to Lightroom on my MacBook. I take advantage of Lightroom's option to make a backup as it imports, to an external hard drive. I also have Lightroom set to back up its catalog to the external drive daily. When I fly I keep the MacBook and the external drive separate to improve my chances of not losing both!

I also separate the cameras (and their chargers and other accessories) so even if one bag gets lost I'll still have a usable camera. I worry as much about the chargers since they'd be impossible to replace in most places and without them the camera would soon become useless. In this respect I like the use of standard AA batteries in my Canon S3. I still use rechargeables but I can use readily available disposables in an emergency, and there's also a much better chance of replacing AA rechargeables and recharger.

After I download I do NOT erase the memory cards and I rotate them so I put in the least recently used card. That way the cards provide a third short term backup in case I mess up an import. I erase a card when I put it into the camera. The only drawback to this system is that occasionally I forget to erase the card when I put it in the camera. I then run out of space soon after I start shooting, but I can't erase the card at this point because I've put new pictures on it. At least Lightroom has an option to ignore already imported photos.

My primary Lightroom catalog and photos are on my desktop system at home. For each trip I start a new catalog on the MacBook just for that trip. When I get home I import that catalog into my main system. One minor drawback of this is that I start fresh with keyword tags and sometimes they don't match with the ones I'm using in my main catalog. But this is minor and easily fixed using Lightroom's ability to rename keyword tags.

I keep both the main and the MacBook catalog organized primarily by date with the yyyy/yyyy-mm-dd format. I rename as I import to yyyymmdd - filename. When I first starting using Lightroom I didn't rename but the straight camera generated file names aren't very helpful, and there's always the risk of duplicates. In my main catalog I add another level of folders for major trips e.g. 2008/2008 Baja/yyyy-mm-dd. This makes it easier to select all the photos for a trip e.g. to put together a slide show.

Some people keep their photos in folders by subject or type rather than date. To me recording subject or type is better done using tags. Otherwise you're continually faced with questions like which folder do I put a picture of a bird on a beach at sunset? In birds, beaches, or sunsets? That's the power of the ability to apply multiple tags, in this case birds, beaches, and sunsets. And it's just as easy to click on a birds tag as it is to click on a birds folder. Of course, this assumes you're inside Lightroom. Outside Lightroom you'd have no easy way to select the birds tag.

I'm not very systematic about my tagging. I'll do a little tagging when I first look at the photos. And I might go through and apply a tag (e.g. birds) if I want to make a selection of a certain type or subject. But lots of my photos aren't tagged at all, which makes it hard to find them later. I could do better in this area.

Another choice is whether to delete "bad" pictures e.g. badly exposed or out of focus. Personally, I don't delete, no matter how bad the photo is. I just flag it as "Rejected" and normally I filter rejected photos out of view. This is similar to Gmail's philosophy of keeping everything. Storage is cheap. It's not so much that you'll ever want a totally ruined photo, it's the time and mental effort needed to decide on the marginal cases. You can always change your mind about a Rejected photo, if you delete it, it's gone.

One case where I might be tempted to delete is when I'm bracketing exposures. Since they're more or less duplicates, why not just keep the best exposure? But again, it's not always clear which is "best". You might actually want to combine several exposures for a HDR image. And Lightroom's ability to "stack" photos avoids having to look at the "duplicates" most of the time.

I'm finding the K10D tends to overexpose bright scenes like snow or sand. I need to figure out when I need to override the exposure, by how much, and what the best way to do that is. I've been bracketing when I suspect there will be a problem. So far it looks like -1 stop does the trick. Lightroom's Recovery slider (and shooting raw) lets you fix a certain amount of overexposure, but it can't recover what really isn't there. This is where composing in the display in the small camera helps. I need to get in the habit of reviewing photos on the K10 and maybe even checking the histogram. Of course, that only works for static subjects where you have the option of another shot.

I also find that for certain shots e.g. closeups I need to manually focus the K10, especially when low light means a small depth of focus. The auto-focus is pretty good, but it can't read your mind and doesn't always pick the right thing to focus on.

On my main system I use Chronosync to automatically back up my photos and catalog to a network drive. And I also have TimeMachine running to an external drive. But currently I don't have a good offsite backup system. I keep a lot of my photos on my work computer, but I don't have a convenient (or automatic) way to do this so it's not too reliable. I've tried a few ways to sync my home and work computers (especially my photos and music), but it hasn't been too successful, partly because of the sheer volume of data to sync, especially at the start. An online internet backup is another option, but again, it's tough with the volume of data, again especially at the start. And not just at the start - when I come back from this trip I'll be adding 40 gb in one shot. That's a lot to transfer over the internet.

When traveling, I try to upload photos to the web every few days so my family and friends can keep up with me. So far I've been using Google Picasa Web Albums. Before Lightroom I was using Picassa to organize my photos, so it was natural to use it's web albums. It works well enough so I've just kept using it. And the Lightroom plugin to export directly to a web album makes it very easy. Now I keep thinking I should give Flickr a try.

I do a certain amount of tweaking of the photos I choose to post - mostly cropping and exposure. For this, and for general organizing, it's great to use the same software (Lightroom) traveling as well as at home.

You can check out some of the photos I've posted at : http://picasaweb.google.com/apmckinlay

Publishing Video

Everyone is posting their video these days, how hard can it be?

I shot some short video clips of a school of dolphins on my Canon SD700 IS.

The first hurdle was to combine them together. I fired up iMovie for the very first time and managed to put in the clips, crop them, and add transitions. So far so good.

I exported the movie and uploaded it to YouTube. The QuickTime mov file was about 100mb for 30 seconds so it took a long time (almost 2 hours) to upload it over the hotel DSL. It would be nice if you got some progress feedback - I started to wonder if it was working or not.

Finally I can play the YouTube video. Yuck. Shrunken down from 640x480 to YouTube size and highly compressed, it looked like crap.

I start looking for alternatives and I decide to try Google Video. One of the reasons is that it recommends 640x480. I try the Google Uploader this time. It still takes forever. And the Google Video version still looks like crap and it appears to have shrunk it even smaller than YouTube (pick "Original Size" on the pop up menu).

The most recommended format seemed to be mpeg4 so I started to look for a way to convert the file. One of the suggestions I found said to use iMovie. Sure enough, although it is far from obvious that iMovie can do this, there are instructions in the help. The key is to choose "Expert Settings". I guess only experts would want something other than QuickTime! It took some trial and error experimentation to get a file that looked decent. I ended up choosing Lan/Intranet, even though that isn't what I want. (I hate it when the "right" settings are obviously "wrong".) This mpeg4 file is under 6mb, a little better than the "default" QuickTime file.

Next I decided to try blip.tv only because I'd recently seen some videos published there. Their FAQ doesn't really say much about resolution or compression, but then again, neither do YouTube or Google Video. Their web upload gives good progress feedback - very nice. (And I'm at the coffee shop this time and the upload rate is better.) I watch the video and, eureka, it looks good. But it says it's still being processed (converted to Flash). When it finishes I watch the Flash version. But at least you can choose the format and if you pick the MPEG4 version the quality is better. I'm not sure if that's a limitation of the Flash format or if they just choose a higher compression. At least even the Flash version is the full 640x480 unlike YouTube and Google Video.

Google actually has a third option - uploading to Picasa Web Albums. But you have to use Picasa to upload video and they don't have a Mac version. I could have installed Picasa inside my Windows VM in Parallels but that seemed like too much hassle.

I'm a newbie at publishing video, and probably pickier than most people. But it seems like this technology is still pretty rough around the edges. Maybe there are ways to make YouTube or Google Video work better. Oh well, in the end I accomplished what I set out to.

Remote Access with LogMeIn

I've been using LogMeIn to access my office computer while I'm traveling (to get access to things that are internal to the office). So far it's worked well, even accessing a Windows machine from my MacBook.

The only problem I've run into is that the Java file transfer doesn't work under Firefox - it locks up so you have to kill Firefox. This is mentioned on their support forum, but so far no solution. It works from Safari though, so it's not a big deal.

The basic version is free. I decided to pay for the Pro version since it was relatively cheap and I could use some of the Pro features.

There's a bit of a lag in typing, but that's not surprising considering I'm using wireless to dsl from Mexico to Canada. Occasionally it trips me up, but for my usage it's fine.

First Time Skype

I finally got around to using Skype to talk to Shelley back in Saskatoon, Canada from Loreto, Baja Mexico where I am currently.

The first time we connected we couldn't hear each other, although we both heard the "ringing" in our headsets. At least we had the Chat facility to try to debug the problem. It was a little harder since I was on the Mac version and Shelley was on the Windows version, so the menu options are different. On the Windows version the settings are in a "Personalize" option - which seems like a rather odd choice of name. (On the Mac it's the standard Skype > Preferences)

Everything seemed ok so we tried again and this time it worked. Not sure why it didn't work the first time. The only thing I can think of was that Windows hadn't finished recognizing the USB headset (Shelley had only just plugged it in).

I could hear Shelley great. She said my sound quality wasn't so good. That might have been due to my bluetooth headset. The microphone is quite far from my mouth. And on top of that, I was outside on the deck (to reach the wireless) and there was a breeze and traffic noises. There's a definite lag time and you pretty much had to take turns talking, but it was definitely usable.

I wouldn't want to use it as my regular phone, but it beats dealing with long distance when I already have an internet connection.

Pretty amazing - bluetooth from headset to laptop, wireless from laptop, and then by internet across North America. And all for "free" :-) And more amazingly, given all the software involved, it works.

You Weren't Meant to Have a Boss

The latest essay by Paul Graham.

I've been my own boss for over 20 years now. And the only job I had before that was for a pretty small company where I had a lot of freedom. Nowadays, I can't imagine working for someone else - it's a frightening thought.

On the other hand, I've known lots of programmers (and other people) that want nothing more than to settle into a secure job in a large organization. To each his own, I guess.

Customer Service

I've been traveling recently. Travel gives you a great perspective on customer service. You're continually dealing with airlines, restaurants, hotels, etc. One of the things I've noticed lately is that you can divide people/companies into two types. One type wants to go by the book. If you ask them for something that doesn't fit into their predefined "boxes" their response is "sorry, can't help you". They might be really friendly about it, but they don't help you. The other type will look for a way to help you, even though you're asking for something unusual or something that isn't really their business.

For example, we go into several agencies and say we want to go out snorkeling to a different island than the usual one. Most of them give us the official response of "we don't go there" or "we don't have enough people who want to go there" and that's the end of it. But then we go to another agency and they say "sure, we can work something out, it may depend on the weather and what else we have booked, but we'll figure it out". That's the kind of service that brings me back for repeat business.

I just hope that my business is more like the second type. Obviously, you can't answer every single request but you can try to come up with something to offer. My staff will come to me and say "so and so wants xyz, what should I tell them". If possible, I'll say "give it to them". Lots of times it's not possible, and you can tell that the staff person just wants to say no. But often we can offer a partial solution or an alternative or a suggestion. It may or may not totally satisfy the customer, but at least you've tried.

Here's an interesting video from a customer service conference: (note: it's an hour long)

Day 4 at Etech 2008

Started out with XO Laptop Hacks. I just got my OLPC XO not that long ago and I haven't had time to do much with it so it was interesting to hear more.

Next I went to the presentation on CouchDB. It's an interesting project - written in Erlang, primary interface JSON via HTTP, started by one guy on his own, but now IBM pays his salary.

I couldn't miss Jeff Jonas's talk on behind the scenes in Las Vegas - he's an entertaining presenter. His talk didn't have as much software content as last year, but it was still good.

Synthetic Neurobiology was largely on the theme of "body hacking". One of their more clever hacks was to introduce genes into neurons (via a virus) that make the cell sensitive to light. You can then trigger the neurons to fire (or suppress them from firing) using light. Wow.

The day (and the conference) ended with more keynotes. First by Alex Steffen of WorldChanging (who I was disappointed to see carrying a disposable plastic bottle of water!) Then a presentation on Twine - yet another social network app, but at least with some interesting semantic web aspects. A brief talk on Digital Democracy about the use of the web (and social networks, of course!) in politics.

And the final talk was by Timothy Ferriss of the (4-Hour Workweek). I'd seen Tim around during the conference, attending talks. My first impression was that he was short. A bit like seeing an famous actor and finding out how short they are. He also blended in pretty well carrying his backpack around, although without the ever present laptop of most people. But when he gave his talk he definitely had a bigger presence and charisma. Unlike every other speaker, he did not have any powerpoint slides - he just talked. If you've read his book and blog it was nothing new. Despite being a wing nut in many ways, I still find him "inspiring" if that's the right word. The gist of his message is that you don't have to be powerlessly overwhelmingly "busy". You can re-engineer your life to do what you want to do.

Day 3 at ETech 2008

When I saw the first speaker of the day was some old guy instead of the scheduled Kathy Sierra, I was a little disappointed. The "old guy" turned out to be John McCarthy, the inventor of Lisp and "artificial intelligence". To me it was a pleasant surprise to see someone like this along with all the "tech kiddies". Unfortunately, his talk moved really slowly and they had to cut him off. I felt bad for him.

Kathy Sierra did speak later. She talked about how we all want to be "good" at something and what it takes to do that. Surprisingly, natural ability counts less than sheer focus and concentration. Of course, that requires motivation and it's less clear how to get that. Lately I've been thinking about when I developed Suneido and my focus (perhaps fanaticism would be more accurate). I seldom achieve that kind of focus anymore and I miss it. I just haven't quite figured out how to get it back. Then again, do I want to get it back? Spending the majority of your waking moments working on something isn't exactly a balanced life, no matter how challenging/rewarding/addictive it is.

Looking at today's sessions it seemed like there wasn't that much that was attractive to me. But sometimes it's a good thing to be be forced to go to talks that I might otherwise have skipped. After all, the whole point of the conference (for me) is to get exposed to new ideas.

Brain Imaging and I Sing the Body Electric surprisingly turned out to have a lot of common ground. There's a definite feeling around that hacking humans is the next frontier. I've never been keen on the idea of "hacking" myself, maybe because I know how easy it is to screw up complex systems.

Hackers Built My Motorcycle started out by saying his talk had nothing to do with the title, and sure enough he never mentioned motorcycles. Maybe that was an example of "hacking" the conference schedule. Nevertheless, it was a fun talk. The problem with talks about security, like talks about the environment, is that they are somewhat depressing! He proceeded to explain how easy it is to hack into cell phones, web sites, house locks, RFID credit cards ... scary stuff.

The next talk, about technology in Cuba, had the potential to be quite interesting but the two speakers read a pre-written speech and that seldom works well. I've really found that a good presenter is worth going to regardless of the topic, and no matter how interesting the topic, a bad presenter will kill it.

My final session was on OpenCV, an open source computer vision library. I don't know much about this area but it was pretty amazing what's possible these days.

Day 2 at ETech 2008

Saul Griffith's talk on Energy Literacy was well done but pretty depressing. I can't dredge up much optimism that a) people will massively reduce their energy use, and b) we'll shut down much of our fossil fuel energy production and replace it via a massive construction of new green energy sources. I can't see it happening. People won't take such drastic action until there's a crisis. So it was further depressing to hear that the time lag on carbon reduction affects can be hundreds of years. i.e. By the time there's a crisis it'll be way too late to do anything. Of course, it's a contentious subject. Someone at my lunch table argued that technology would save us. Hmmm ... maybe it'll save us humans (we're good at that) but what about the rest of our ecosystem? We don't have near as good a track record there.

The presentation by MegaPhone on Collaborative Gaming in Public Spaces was entertaining. The two presenters were the founders and looked to be still in their teens! They develop ways for people to interact with public video displays, primarily by cell phone. They ended up in the debugger on the big screen a few times, but it seemed to work in the end. I'm not sure how they got to be keynote speakers but it was refreshingly "innocent".

The session on the Future of Mind Hacks was pretty interesting. You always wonder if you've picked the right session, but when Timothy Ferriss is a couple of rows ahead and Tim O'Reilly is a couple of rows behind I figure I've picked a good one.

Two Microsoft employees were sitting beside me, one had a MacBook, the other an iPhone (and worked in the mobile division). Hmmm ... is that getting to know the competition?

After the keynotes I went to Tap is the New Click on gestural interfaces (e.g. the iPhone). It was ok, but not great, mostly just a dry overview.

At lunch I counted 10 tables with a single person at them (one of them was mine). Good to know I'm not the only anti-social geek. But most geeks must be more social electronically than me. One of the big topics is social applications. Frankly, I don't have enough friends to need an application to keep track of them. Then again, I don't even have a cell phone so I'm obviously abnormal.

After lunch I went to Green Nano by an HP researcher. I've been excited by nanotech since Eric Drexler's Engines of Creation so it was nice to hear about progress. But the talk seemed a little dry.

For some variety I next went to a talk on Digital Activism by Ethan Zuckerman. It turned out to be pretty interesting. It's good to hear about some "positive" uses of technology.

Then DIY Drones by Chris Anderson (of Long Tail fame) - fun.

And finally, Personal Productivity by Gina Trapani of LifeHacker, another good talk. (Of course, Timothy Ferriss was at this one as well.)

All in all, a pretty good day.

Day 1 at ETech 2008

The first day at ETech is tutorials.

First up was Live, Vast and Deep: Web-native Information Visualization by Tom Carden of Stamen Design. Although it was pretty high level and didn't get into too many details, there were lots of thought provoking examples and links to things to investigate - and that's what I'm looking for.

It was a tough choice between this and Storyboarding for Nonfiction by Kathy Sierra (or Creating Passionate Users). I decided that I might get more useful ideas (for me) from visualization. But I bet Kathy's talk was good too.

In the afternoon I went to Debugging Hacks: What They Never Taught You About Solving Hard Bugs by Marc Hedlund of Wesabe. Nothing really new but a good talk on how to solve hard bugs. One point that resonated with me: "the goal is not to suppress the symptoms, it's to understand the problem". I have this discussion with my own programmers on occasion - removing an assert is not "solving" the problem! And conversely, adding the assert did not "cause" the problem.

San Diego ETech

I'll be in San Diego this week for ETech.

Drop me an email if you're in the area and interested in meeting up.