Phone Companies Antics

I had a phone call from Rogers yesterday trying to sell me a "Rocket Stick". I asked if I could use my existing data plan. "Oh, do you have a data plan?" they replied.

I wish these companies would get their act together and use the information they have. They say they're calling me because I'm a "valued customer" but they don't even have my account information. Of course, it's probably a different department or outsourced or one of a dozen other excuses. But it makes them look stupid. One of the first rules of selling is to know your customer.

So they asked me what my plan was ($30 per month for 6gb) and told me "Yes, you can switch to a shared plan". I was pleasantly surprised because I assumed the answer would be "no".

The new plan would be $45 per month for 1gb. Wait a minute, you want me to pay 50% more for 1/6 the service? "Yes, but it would be shared."  No thanks, doesn't sound like a good deal to me. (I don't use 6gb and 1gb would probably be fine, but I still don't want to pay more for less! And with 6gb I don't have to worry about my usage, which is a big plus.)

Actually, what I'm really interested in is the Mobile MiFi Hotspot. Then I could use it with my laptop and Shelley would use it with her iPod Touch or laptop at the same time. (Plus any future wifi devices.)  I'd looked this up before, but it just said "Out of Stock". I asked about it and they told me they used to have them, but there was some problem with the batteries. That sounded a little lame to me - they're still selling them in the US.

Note: In theory I can share my data plan by tethering my MacBook to my iPhone but it hasn't worked too well when I tried and I haven't spent the time to figure it out. And I don't think that would help with other devices.

see also: AT&T’s cynical act

Impossible Things

"Why, sometimes I've believed as many as six impossible things before breakfast." Alice in Wonderland

I was testing jSuneido on a Windows system and I got:

java.io.IOException
        at java.nio.MappedByteBuffer.force0(Native Method)
        at java.nio.MappedByteBuffer.force(MappedByteBuffer.java:154)

So I went to add a try-catch to log this error. But Eclipse told me MappedByteBuffer.force doesn't throw IOException

I check the documentation but it didn't mention IOException either.

This is a "checked" exception, meaning you can't throw it without declaring that you throw it, and you can't call something that may throw it without catching it.

But I guess all that goes out the window when you get down to native methods.

I just caught the more general base Exception instead.

PS. I wasn't able to recreate the error, I just hope it won't come back to haunt me.

Apple Overtakes Microsoft in Market Value

Apple Overtakes Microsoft in Market Value: End of an Era?: "At the close of Wednesday’s trading, Apple was valued at $222 billion, while Microsoft was worth $219 billion. Apple’s shares ended the day at $244.11, while Microsoft’s finished at a seven-month low of $25.01. And it isn’t only Cupertino’s successes, but also Redmond’s failures that are responsible for the new power dynamic between the two companies. Overall, Microsoft stock is down 20 percent compared to 10 years ago, while the value of Apple’s has grown tenfold over the same period."

Motivation

A great animated video for a TED talk by Daniel Pink related to his recent book Drive.

Third Time is the Charm

I decided that for what I needed Netty (or Mina) was overkill.

So I rewrote my server code using thread-per-connection and blocking io. I still used NIO Channels because I wanted to be able to use gathering writes (and because it was closer to the previous code).

And I still had the same problem!

That tells me it probably wasn't a mistake in my NIO non-blocking Selector version.

And the same fix "worked" for this version. (synchronizing the SocketChannel write)

That seems to indicate the problem is really in SocketChannel.write (or ByteBuffer)

I posted a question on the network forum on Sun (or should I say Oracle) but so far no response.

So I rewrote the server code a third time, using just old style java.net Socket, no NIO, no channels.

This time it seems to work!

On the positive side, the blocking, thread-per-connection versions (with channels or sockets) seem to be about 15% faster than the non-blocking Selector version.

Assuming I don't find any problems, and depending on what (if any) response I get to the forum question, I'll probably go ahead with this version.

Not Making Sense

Two more days working on my concurrency bug. I'm growing less and less confident that I'll "figure it out". The more I narrow it down, the less frequently it happens, which makes it more and more difficult (and time consuming) to know whether I've "fixed" it.

I'm really reluctant to say it, but I'm starting to wonder if there's a bug in Java. I have it narrowed down to where I check the data on the server immediately before writing it and it's always ok. But every so often the client will receive garbage. (Which usually leads to a timeout because it's waiting for a newline.) I'm pretty sure it's not a problem on the client side because it only happens when the server is multi-threaded.

If this was C or C++ I'd assume I had a bad pointer and was overwriting memory or that I was referencing memory that was prematurely free'd. But that's not supposed to be possible in Java.

There is no concurrent access in my code to the data that is being sent. I don't reuse the buffer or anything like that. I've tried using a read-only buffer, and I've also tried allocating a fresh buffer every time.

I have found a "fix". If I synchronize the channel.write on a global object, so only one thread can write at a time, then it seems to work. At least I've run it for about 10 times as long as it usually takes to get a failure.

But this "fix" makes no sense. First, channel.write is supposed to be thread safe. Second, concurrent writes (when they happen) are to separate channels, channels are not shared by multiple threads.

It could be a concurrency bug in Java or the OS, but why would I be the only one to encounter it?

Of course, this change may not be directly related to the problem. It may just be altering the timing enough to make the bug not happen, or to happen so rarely I'm not seeing it.

I've gone back and re-read all the NIO material I can find on the web. Despite a certain amount of conflicting advice I think I have a pretty good idea how it works, how to use it, and the common errors. I even read the arguments against using NIO (which I'm sympathetic to, given my problems).

Although it seems to be working, I'm not comfortable leaving it as it is. Since I don't know the source of the problem, or why/how the "fix" works, the bug could reappear at any time. But I'm also running out of things to try. It's narrowed down to only a few hundred lines of fairly straightforward code, most of which is identical to examples found on the web.

So my options are to abandon NIO and use a simple thread per connection model. According to some reports, this would be faster for the number of connections I expect (up to several hundred).

Or I could use someone else's framework. The contenders seem to be Apache Mina and JBoss Netty. (see Netty vs Mina)

I think I'll probably give Netty a try and see how it goes. But it's frustrating not being able to figure it out - I hate "giving up"!

Be Precise

Guest Post by
Penny McKinlay

I’m sure all of us have used the Help feature on a software program and been frustrated and annoyed because it didn’t make sense or didn’t answer our question.

One of my goals as a technical writer is to develop a good relationship with the customer. I want them to turn to the documentation feeling confident that it will help them complete their task quickly and successfully.

I don’t want to confuse or antagonize them.

Avoid Confusion

I prepare the online documentation for customers of an integrated trucking and accounting software package. I used to use the verb “select” when telling customers to pick the appropriate option on a pull-down menu.

Fortunately, one of the trainers pointed out to me that this was really confusing for the customers because there was a Select button at the bottom of many of the screens. Now, I only use the word “select” if I am referring to this button.

The Help carefully distinguishes between “placing a checkmark in the checkbox” and “generating a check.” And we deliberately use the American spelling of “check” so as not to irritate or confuse American customers.

We consistently refer to “Owner Operators” rather than switching back and forth between “Owner Operators,” “Leased Carriers,” and “Third-Party Contractors.”

Don’t Annoy the Customer

I try and write the Help from the user’s perspective: “you will not be able to invoice your Order until you do this” or “use this report to review outstanding trucking transactions.”

We used to say that the software would “allow” customers to do certain things, but we realized that the word “allow” was really condescending. Now, we use phrases such as “this screen will help you to . . . .”

We also try very hard to avoid jargon because it’s intimidating and makes readers feel stupid. Software programmers talk about “parameters” and “locking down.” The Help talks about “options” and “protecting.”

Writing Tips

Be precise, be consistent, consider your audience – good things to keep in mind whatever we’re writing.

For more tips on business writing, visit my web site.

Penny McKinlay
pennymckinlay.axonsoft.com
www.wanderlustandwords.blogspot.com
twitter.com/PennyMcKinlay

enTourage eDGe™

enTourage eDGe™

Another interesting alternative to the iPad or Kindle. I'm not sure it's ready for prime time though - it sounds like it's slow, heavy, limited battery life, and no apps. Other than that, it looks great :-)

I'd really like to go for something more "open" than the iPad or Kindle, but the problem is the apps and books. No one has all the apps like Apple, or the books like Amazon.

If we can move toward HTML 5 apps and get away from DRM, then we'll have a lot more freedom with hardware.

Which still leaves the question whether anyone can match Apple in terms of design and user experience. Even people who promote Android admit that it's not as polished as Apple. The software might get there eventually - Linux (e.g. the latest Ubuntu) is starting to rival Windows and OS X. But Apple's hardware design and polish is tough to match.

iPhone / Touch Apps

I gave my sister Penny an iPod Touch for her birthday and figured I'd better give her some recommendations for apps. Here's some of what I have on my iPhone:

Evernote (free) - save notes, photos, documents, works with web, Windows, and Mac versions
BlogPress ($2.99) - blog post writing tool, you can write offline
Note Taker ($1.99) - hand write notes with your finger (free Lite version)
Kindle (free) - ebook reader, works with Kindle devices or Windows and Mac software
Tripit (free) - trip itinerary, syncs with web site
iKeePass ($0.99) - password manager, works with web, Windows, and Mac versions
Remote (free) - remote for iTunes on Mac computers
Facebook (free)
Echofon for Twitter
Encyclopedia ($8.99) - offline copy of Wikipedia
iBird (free to $29.99) - bird guide (search for iBird)
GoodReader ($0.99) - PDF reader (free Lite version)
Instapaper (free) - save web pages for later offline reading (Pro version $4.99)
Outliner ($4.99) - syncs with web site
Google Earth (free)
oMaps ($1.99) - open maps
Fugawi iMap ($4.99) - topo maps of Canada and USA, can download maps for offline
Copilot Live Directions (free) - offline street maps of Canada and USA
Stanza (free) - ebook reader
Eucalyptus ($9.99) - ebook reader, access to 20,000 free books
Dropbox (free) - works with Windows and Mac software and web site
Gnu Go (free)

Perseverance

Another day working on my bug but at least I've made some progress, at least as far as narrowing it down.

I managed to simplify my test to just repeating the same request over and over (instead of the semi-realistic mix of operations in the original test) This greatly reduces the amount of code where the bug could be.

Next I wrote a test client in Java to do that same request. But it succeeded. (Of course, you never know if one more run would run into the bug.)

Doing more testing with the cSuneido client I realized that the bug occurred more often (or only) when the client and the server were on separate machines. So I tried running my Java client on a separate machine. And now it failed too.

So now I know the problem isn't with cSuneido, or Parallels, or Windows.

I narrowed it down even further by commenting out all the real work of the request (the database querying stuff) leaving only the framework of request handling. It still fails. This reduces the amount of code where the bug could be even more.

But I'm still puzzled. My simple test client plus test server has yet to fail, even running on separate machines. But the test client plus the real server does fail, even though the real server is cut down to where it's doing little more than the test server. There must be some difference. Or else it's just that the probability of the error is less and I haven't run it enough times.

Just to give an idea, when it's failing, it fails about once every million requests. Gotta love that kind of bug!

So I still haven't found the problem, but I might be getting closer.

This is what I was afraid of!

I just spent another day trying to find my jSuneido concurrency bug, with no concrete progress. That makes four days I think, definitely the trickiest bug on this project (so far). It's this kind of bug that makes me afraid of concurrency in the first place.

I made a test server and test client and at first I thought I had recreated the problem, but the fix I found only worked for the test program, not jSuneido :-( Since then my test server and client have worked perfectly, despite extending them to be more like the real jSuneido code.

I've tried synchronizing here and there and making defensive copies of this and that but the bug still happens. I've read and re-read the code. I've drawn diagrams of the sequence of events.

I start to wonder if the bug is in the cSuneido client rather than the jSuneido server. But that doesn't really make sense because we have thousands of people using the cSuneido client and we don't see this problem.

My next step is to write a simple Java client for the jSuneido server. That will eliminate the cSuneido client. I think the best bet is to try to gradually narrow down the problem and isolate it.

I don't doubt I'll find the problem eventually. The question is how many days I'll need to bang my head against the wall. Hopefully the wall crumbles before my head!

People Hate Change

My company sells specialized software for trucking companies. Often companies that we sell to have been using QuickBooks. And the people in those companies are very reluctant to leave QuickBooks and switch to using our software. And even after they do switch they continue to complain that our software does not work like QuickBooks.

QuickBooks is a good program and no question there are QuickBook features that are our software lacks. But for trucking companies our software does a lot more than QuickBooks ever will. QuickBooks doesn't handle the specialized needs of trucking companies, our software does. But to the people being faced with change, none of that matters, they want the familiar.

This was brought home recently by a long time customer of ours. They're not a trucking company and the software they have is very old. (It's DOS text mode software!) Recently they wanted to set up another company and wanted another copy of this ancient software. We suggested they just use QuickBooks. They came back and said they hated QuickBooks and that their current software is much better! This is so obviously not true that it was funny. It was a good reminder of how strongly biased people are to what they are used to.

jSuneido Concurrency Bug

I spent all day yesterday working on my concurrency bug in jSuneido. (see previous post)

It didn't seem like I made any progress, but I guess that's not true - I eliminated a few possibilities.

Originally I assumed it was a deadlock since it was "freezing". So I managed to make the problem happen (that takes a while), connected JConsole, and ran the deadlock detector. It didn't find any deadlocks. Thinking about it, that makes sense since it's really only the client that gets "stuck" - the server thinks everything is fine. There are no worker threads running for that client.

So much for that assumption. I tried to narrow it down, without much success. I added a bunch of assertions, which failed, but I found it was my assumptions that were wrong, not the code.

The client appears to be waiting for a response to a request. But the server is not in the process of handling a request - it either didn't get the request, or it thinks it finished it. The read and write buffers for that client are empty.

Sometimes you have to run test for a long time before the problem shows up. So you try something, and the longer the tests run successfully, the more you think you've got it fixed. But eventually it has always shown up. I guess I should be thankfully this is on the order of minutes rather than hours! Even so, it's reminiscent of the "old" days when you had to have something to read while you waited for compiles. (And no internet to browse back then.)

On the positive side, the handling for inactive transactions and clients seems to be working well - when the client gets stuck, the server eventually aborts its transactions and if it stays stuck, closes the connection.

My current hypothesis is that it's related to a potentially overlap between handling requests. As an optimization, worker threads first attempt to directly write their response. This is in non-blocking mode so it may or may not write everything. Any remaining data is sent by the select thread when the socket becomes write ready. But if the worker does successfully write the entire response, and then there is a context switch back to the "select" thread, the next request can be read and another worker started, before the previous worker has a chance to finish. I'm not sure why this would be a problem, but my assumption when I was writing the code was that workers for the same client would not overlap, so I didn't worry about making the data structures thread safe. That's the next area I want to look at.

Dig Deep

In the last few days I've questioned a number of changes that I saw going through version control. None of them was "incorrect". The recurring theme was that they were fixing problems at too "shallow" a level.

For instance a dialog was using a feature to save its size after being changed by the user. This wasn't working properly, so the feature was disabled. There's a couple of problems with this. One is that we lose a feature from that dialog that we presumably wanted. But more importantly, that feature is used in many places and if there's a bug in it, it's going to show up elsewhere. Are we going to disable it everywhere? How much time are we going to spend dealing with each new instance that turns up?

Of course, sometimes you have to fix a bug the most immediate way. That's fine as long as you go back and look for the real problem.

Here's another example: we got an error about a missing database table. This was after wiping out all the data so the initial thought might be to ignore it. But this table is supposed to be created automatically. So we dug a bit deeper and we found one place where it didn't create it when necessary and fixed that. But access to this table is supposed to be encapsulated in one class. This class handled creating the table if necessary. The spot we fixed was accessing the table directly. So we moved this access into the class to restore encapsulation. But then we realized that it shouldn't even need to use this class/table. It turns out we had used it to work around yet another bug. So we removed the use of the class/table and fixed the original, previous bug. The end result was less code and simpler.

This process of digging deeper into a problem can take time. It can take orders of magnitude more time than a quick fix to the immediate problem. Which is why we often apply the quick fix and move on. But the long term result of that approach is technical debt - your code gets more complex, less understandable, less consistent, hard to change. And in the long run the quick fix ends up costing your more time than if you had fixed it properly in the first case.

This is a bit like the "lean" idea of finding and fixing "root causes". One of the techniques is to ask five (more or less) "whys". Why did we get a missing table error? Because the code wasn't creating it on demand. Why? Because it was bypassing the access class. Why? Because it was doing something non-standard. Why? To work around another bug.

Is It All About Sales?

It always bugs me when people (usually sales people) say "It's all about sales. Everyone in the company needs to be selling all the time, not just the sales and marketing department."

Before anyone gets the wrong idea and starts jumping on me - I am not disagreeing with this. But I don't think it's the whole picture, and I don't think it's the most useful way to think about it.

Take my company - we have sales and marketing, product development, and customer support. To me, they're like three legs of a stool. Does it make sense for one leg to say it's all about that leg? All three legs are essential. Take away any one and the stool will fall over.

Part of the reason sales is seen as primary is that they bring in the money. It's pretty obvious that's an important role. They have the most direct link to the success of the company. But just because the other roles have a less direct link to success doesn't make them less important.

Think of fighting a fire. The fireman holding the nozzle of the hose has the most direct link to spraying water on the fire. But without the rest of the hose, and the source of water, he would be out of business.

Take Apple for example, obviously great at sales and marketing. But are they "all" about sales? Where would they be without great products?

A good marketing and sales group sometimes think it can sell anything. (And there is some truth to that.) But when my company has tried to sell third-party products it has become pretty clear that if the product doesn't have good development and good support then in the long run, it's pretty hard to sell.

You don't usually hear "it's all about customer support" or "it's all about product development". To me, those views would be just as incomplete. For example, 37 Signals claims they have no marketing people. That may be true in the sense that they don't have anyone whose only job is marketing. But get real, those guys do more marketing than most marketing departments. They're certainly not "just" programmers.

Perhaps another reason you hear "everyone needs to sell" more often is that non-sales people tend to avoid selling. Whereas sales and support are usually more than happy to talk about where the product should be developed. Another weak spot is that programmers tend to avoid customer support (I know I did!).

While equally incomplete, I think there's value in the idea that everyone needs to do customer support and everyone needs to do product development, just like there is value in the idea that everyone needs to sell.

I think what it boils down to is that everyone needs to keep the big picture in mind, not just any one particular role. That's easier and more obvious in a small company. Most of the time I think my company does fairly well at this. Sales and support contribute to product development, programmers and sales help with customer support, and yes, hopefully everyone helps sell.

Cultures and Attitudes

You would think that peoples' attitudes and behaviour would be an individual thing with a range of variation in any organization.

But attitudes and behaviours are greatly affected by the culture of an organization, biasing people within the organization to act a certain way. There is still variation, and you can find good people in a bad organization (or vice versa). But even this is reduced by a tendancy for good people to leave a bad organization, and bad people to be removed from a good organization.

Amtrak (US passenger rail) seems to have a culture of small minded bureaucracy.

On the way to Portland there were only a handful of people in our car. So one person moved to a seat with more room. No big deal, until the steward came along: Is this your seat? You must remain in your designated seat. It is not possible to switch seats. The passenger wisely kept quiet. Eventually the steward wound down his littany of pronouncements. The passenger looked at him. Then the steward said "Ok, here's what we can do. I can let you stay in this seat as long as you agree to move if someone else is assigned this seat.". I had to shake my head. This was exactly what the passenger had said he would do right from the start. "But you must leave your seat assignment card on your assigned seat."

But that wasn't the end of it. A bit later a second steward came around: Is this your assigned seat? No, but the other steward said I could sit there. You could tell he was frustrated at having his exercise of power cut short and looked for another outlet. "Ok, but you must have your seat assignment card on the seat you are sitting in, not the seat you were assigned." (The exact opposite of the first steward.)

This kind of bureaucratic attitude is often exemplified by people saying "you must" or "you cannot" or "I cannot", even (or especially) when this is obviously untrue.

For example, when I went to get tea I was told "I cannot put hot water in your cup.". Obviously he "could", presumably what he meant was that he wasn't supposed to. So why not say that? It would be a lot less frustrating if he said "sorry, I'm not allowed to do that". Or just bend the rules. The previous time, the person had put the hot water in a paper cup, as he was required to do, and then poured it into my cup. This satisfied his requirement, while at the same time fulfilling my desire to not waste a cup.

I think creatively interpreting (bending) the rules (and not being punished for it) is a sign of a good organizational culture. Whereas, presenting the rules as "cannot's" is a sign of a bad one.

The problem is that saying "you cannot" makes you feel more powerful. While saying "I'm not allowed" makes you feel less powerful. And of course, most people would prefer to feel more powerful.

I worry about where my own company's culture fits. I hope it's clear that people should understand the reason for "rules" and use their judgement in interpreting them. And at the very least be truthful and say "I'm not allowed to do that" rather than "you cannot".

Building the Wrong Thing

"The biggest cause of failure in software-intensive systems is not technical failure; it's building the wrong thing." - Leading Lean Software Development

(Presumably that assumes a certain minimal level of technical expertise.)

I spend a lot of time thinking about how to reduce the errors (bugs) in our software. We spend a big percentage of development time on fixing errors. Just to be clear, most of these errors are small internal issues that aren't even noticed by the majority of users. I think as far as our users are concerned, our quality control is good.

But if we could reduce those errors, we would have more time to get stuff done.

But ... if the biggest cause of failure is building the wrong thing, then having more time won't necessarily help. Or at least it's not the best way to improve. By this line of thinking, we'd be better off working on building the right thing, rather than less errors. (Of course, you still want to reduce errors as well.)

It reminds me of the old saying "if it doesn't have to work, you can make it as small/fast/etc. as you like", if you interpret "work" as "building the right thing".

Of course the trick is how to do better at building the right thing. In many ways that's tougher than reducing errors, and that's tough enough.

If you're not involved in the software business at this level, this talk of "building the right thing" may strike you as strange. Why wouldn't you build the right thing? The problem is that no one knows what the "right thing" is. And no, the customers don't know either. As Ford said, "If I'd asked my customers what they wanted, they'd have said a faster horse."

None of this is really new, but it's good to be reminded. Definitely food for thought.

Books and More Books

If you like books and you're in Portland, don't miss Powells. And if you like technical books (eg. computer stuff) don't miss their technical book store a few blocks away.

The technical store even has a display of old computer hardware - an Apple II, an Osborne, an original Mac, etc. I can remember quite clearly when those machines were the hot new thing. That's dating myself, but there's something to be said for perspective.

One of the nice things about Powells is that they mix new and used books. That means you see older books and books that are out of print. They had several copies of Software Tools by Kernighan and Plauger. I remember how excited I was about that book. It came out in 1976, when I was 16. A few years later, a friend and I wrote versions of some of those tools (in C, compiled with Whitesmiths compiler) and sold them. Not that we made any money at it.

That's the exception though. Most computer books don't age very well. They become useless a few years after they're published. I should know, I have piles of them!

Between the two Powells stores, and a Borders I visited on the way there, and between computer, science, environment, and science fiction, I made note (on Evernote) of 30 books I wanted to read. I didn't buy any, which in a way was easier - it would have been tough to choose! (I've already bought one on my Kindle).

Oh well, I guess there's worse things to be addicted to!

Hands on the iPad

I got to touch an iPad today at the Apple store in Portland.

As others have mentioned, it is heavier than you expect. I'm not sure why that is when you've never felt one before. Maybe because the ads make it look so thin. You expect it to be as thin and light as an iPod Touch but it's thicker and heavier like an iPhone.

Beyond that it was like a bigger, faster, fancier iPhone. Cool, but nothing earth shaking.

Do I want one? Sure. Am I desperate to buy one this minute? Not really. My current iPhone plus Kindle combo works pretty well for me.

It was interesting to listen to people's reactions to the iPad. People still aren't quite sure where it fits.

As usual, the Apple store was packed. There always seems to be an undercurrent of excitement.

I fondled the new MacBooks, but externally they haven't changed much. It's that i7 heart I lust after. And I gazed longingly at the 26" iMac with quad core i7. Someday. It's not like there's anything wrong with my current MacBook or iMac. Just seductive marketing.

Location:SW Naito Pkwy,Portland,United States

A Plea to Evernote

Evernote is a popular app for the iPhone and iPod Touch. (There are also PC and Mac versions.)

You can write notes and record photos from the iPhone camera. Notes are synced to Evernotes servers and to all your devices. You can organize and search your notes. Evernote even does OCR (optical character recognition) on images so you can search for text in images.

It's a great product and many people swear by it.

But ...

The software is buggy. It freezes. It crashes. You have to reinstall it periodically to get it working again. Check out the problem reports on GetSatisfaction for more examples.

A tool like this is only worthwhile if it's dependable. I put all my travel information in it and it died almost as soon as I left home. That doesn't make a happy customer. (thankfully reinstalling got me going again)

I'm not the only one with a love hate relationship with Evernote. People give up on it, come back thinking it's fixed, only to discover some time later (after entering a bunch more notes) that it's still got bugs, just a different set from before.

I'm not sure where the problem lies. But if they want to be successful in the long run they have to do better.

Quit adding new features until it's more stable. Add more automated tests. Set up alpha and beta testing progroms. Review the code. Refactor the code. Throw out problem code and rewrite it. Whatever it takes. I can't see anything more important.

It's a good product but it doesn't matter how many great features it has if it's not reliable.