Yesterday morning, I got a brief email saying “Twitter integration does not work. Otherwise, it is a cool app”. The writer was referring to Economy for iPad
This blog-post’s title is the short answer to the question on why Twitter integration isn’t working. See below for a more detailed (and somewhat technical) description of the problems.
The first app: Around four years ago, Twitter announced that they would cut off Basic Auth for Twitter and that all apps needed to use Twitter OAuth. The iPhone Economy app was probably the first iPhone app to use Twitter OAuth for login and posting tweets. I wrote about it in 2009. In terms of development time, implementing Twitter OAuth login turned out to be much more expensive than I anticipated. Twitter didn’t create a mobile-friendly login page until much later and that was a big problem for most users. However, in spite of all these problems, the basic functionality worked and folks were able to post economic tweets with the Economy app (and #cEconomy hashtag)
Twitter breaks the access_token request: Earlier this month, I noticed that the Economy app could not post economic tweets anymore. New logins were also failing. It turned out that Twitter broke their OAuth 1.0 API. Debugging old code isn’t always easy and I had written this code in mid-2009, but fortunately, Twitter made trouble-shooting relatively easy by returning a helpful error message “Required oauth_verifier parameter not provided”. Fixing this problem meant two things (a) After the user types in their username and password, Twitter appends a query-string parameter (oauth_verifier) to the callback URL. This oauth-Verifier value needed to be retrieved from the URL. (b) The oauth_verifier needed to be included in the HTTP headers of the /oauth/access_token request.
Why Twitter broke access_token: Requiring the new oauth_verifier token clearly broke the previous definition of the access_token API and it broke Twitter functionality in third-party apps that relied on the official Twitter definition. Interestingly, Twitter documentation continues to describe oauth_verifier as an optional parameter. Twitter can make the case that their previous implementation and current documentation are both wrong and that incremental security gains weremore important than not-breaking their old API. However, it would have been nice if Twitter had publicly acknowledged their past mistake and given developers sufficient time to migrate to the new requirement. Of course, the best option would have been to require oauth_verifier in the original 2009 API implementation.
The broken request_token API: Current documentation of the request_token API states that the oauth_callback is required. Presumably, this documentation was updated recently. However, interestingly, our old code (without) the oauth_callback parameter continue to work fine. The oauth_callback parameter wasn’t necessary because the callback value is already pre-registered for our app on dev.twitter.com. Nevertheless, I spent the extra effort on adding the new parameter, modifying signature etc. and we now pass the oauth_callback parameter for request_token API calls. Apps don’t get broken regardless of whether the oauth_callback parameter is passed. However, I’d still consider the API as broken because it doesn’t conform to Twitter’s official definition of the API and it could potentially break apps in the future.
No more XML: Twitter seems to have joined the ‘XML is not cool’ bandwagon and they’ve announced that they’ll cut all support for XML response formats in the Twitter 1.1 API. The Twitter 1.0 API will stop working on May 7, 2013. The Twitter 1.1 API also removes endpoints such as /statuses/update.xml (and calls to these APIs will need to be replaced with calls to json endpoints such as /statuses/update.json). It seems unnecessary to cut off XML support and I don’t see this decision generating any real value for Twitter.
Hiding the tweet source: In 2009, Twitter promoted OAuth by announcing that the source (i.e. app) which created a tweet would only be shown if the app used OAuth. This helped users identify the source of a tweet (and perhaps download the app) and it also helped promote the app. Last year, in another flip-flop, Twitter decided that users didn’t need to know anything about the source of a tweet. So the source of tweets isn’t listed on Twitter’s official apps (or website) anymore and users won’t be able to identify apps that generated the tweets of people that they’re following.
Breaking Twitter Search (by source): The Economy app appends the #CEconomy hashtag to all its tweets and this helps create a collection of economic tweets that users can browse through. To eliminate spam, the app only shows #CEconomy tweets created with the Economy app. IOW the search results are filtered by the “source” (i.e. app that generated the tweet) . In 2009, this could be done with a simple HTTP request like http: //search.twitter.com/search?q=%23cEconomy+source%3AAppName. However, a few months ago, Twitter broke this query and the Economy app could not show the list of economic tweets it created. It turned out that Twitter broke this browser query and replaced it with http: //twitter.com/search?q=%23cEconomy%20source%3AAppName. This was an easy problem to fix and less serious than the broken Twitter APIs, but it did inconvenience a lot of users.
What next: The Twitter API has been unstable and besides wasting my time, this instability inconvenienced many users of the Economy app (though I should add that the percentage of users who post economic tweets with the app is fairly low).
I considered cutting the entire Twitter feature, but ultimately fixed the Twitter problems because (a) users who post (or want to post) economic tweets will appreciate keeping the Twitter integration feature (b) the fixes were’t too expensive, didn’t risk other parts of the app and hopefully the Twitter team wouldn’t break more things in the future (c) it is hard to make a totally objective/rational decision about cutting your losses (and killing Twitter functionality) when you remember how much time you had invested in implementing Twitter functionality. I’d like to think that the first two reasons played a bigger role in keeping Twitter integration in the app. In any case, last week, we tested the Twitter integration fixes I wrote and everything works fine.
Next month, we plan to release a major new update for ‘Economy for iPad’ and the Twitter fixes will be included in that release.
Cascade Software Corporation 
That’s too bad. Did they send you any warning of breaking changes? Maybe they tweeted about it and that went unnoticed.
Facebook has been better lately, about how they communicate changes
No, they neither sent me email nor DM. They didn’t even tweet about it.
In fact, they don’t regret not giving any notice, see https://dev.twitter.com/discussions/16443#comment-36721 –
when it was suggested that Twitter should give developers some advance notice, a senior Twitter Partner Engineer replied saying “We wouldn’t make changes to the OAuth flow without notice unless we felt it was necessary.”
IOW when they they make an optional parameter mandatory, Twitter doesn’t think it necessary to give developers any notice. That is adding insult to injury 🙂
On the plus side, I’m glad that someone from Twitter is replying to some questions. Sometimes, a curt reply is better than no reply 🙂
That’s not good developer relationships.
Many companies suck at this, including google. Last night, I wasted a good 3 hours because google analytics APIs don’t behave the way google advertises them. And there appear to be tons of other developers running into the same issue. I had to try brute force and prove that some google APIs don’t accept the same simple auth token as others do. Totally wasted effort.
There are many smaller companies that do excellent developer support though, and some big ones. Like for example, facebook, with it’s complicated API surface, has a lot of infrastructure to take in bugs, answer questions, provide documentation, and notify changes. Stripe, a payment service I use is another great one.
This only highlights the importance of thinking ahead about which third-party API should be use for your product. And explains the bar if you’re thinking of making your own API.
Reminds me of “Twitter lock-in” post by Manton Reece
http://manton.org/2012/08/twitter_lockin.html