SMessenger , formerly known as Twitter Messages, was one of the most successful, arduous, and exciting companies I worked on. I spent 3 years working, planning, thinking, and coding on 3 apps, cross platform video chat, cross platform voice calls, and much more. It will be difficult to encapsulate the breadth of work I put into this company, but I will try to outline our successes and failures along the way.
Part 1: The Genesis
In n 2015 messaging apps were significantly growing in popularity. Reports indicating WeChat’s and Tencent’s dominance were making its way across the world and ending up on the desks of many influential venture capitalists and tech workers. Facebook had just indicated that they were going to separate their messenger app from their main facebook app. Telegram had just begun exploding around the world, and Kik also continued innovating and working on their suite of messenger apps. I began ingesting many of these reports, marinating over the importance and potential of these ideas. It was clear to me, and eventually the rest of the industry, that messaging apps were primed and ready to be built out to the level that Asia already had.
Twitter did provide a very simple and unrefined Direct Messaging experience - but they didn’t seem to want to invest any time or effort into building out the user experience. It is clear now that their infrastructure and API, a relic of repeated missteps by the team, was not set up for future malleability - and so for them to even begin thinking about updating the Direct Messages experience they would first have to revamp the entire API it seems. It was clear to me that they wouldn’t be attempting this for many years - and it seemed like a huge missed opportunity.
It was interesting to note that Twitter, arguably one of the most influential and popular apps in the world, was not pursuing any form of involvement in the impending “Chat Wars”.I already had a lot of experience with Twitter’s antiquated API (through a social media aggregation app I worked on years previously) and was comfortable playing around with it. I quickly concluded that my assumption that Twitter’s DM API’s were not capable - and so I decided I was willing to create my own API layer to access the API with significant workarounds. As I’ve said before, you get a certain level of control from custom built programs and are afforded the ability of freedom from a platform should you run into problems that are out of your control (such as rate limiting, api deprecations, lack of support). This ended up being the best choice for this project, and allowed the company to transform and grow.
Let’s talk about what was so wrong with the Twitter DM API (Note: The API has changed in the last few years and have become a little more robust, but more restrictive as well - I was working with the original DM API). At the time every Direct Message was stores only as one entity in their database. This is poor/lazy practice as it means a shared row between 2 users - not useful for things like read receipts, deleting messages on one users side, among other problems. Also, Twitter’s API only allowed a developer to access 20 sent messages, and 20 received messages per API request - with a limit of 200 previous messages. This was an incredibly restrictive design decision the developers made - so instead of constantly using their api to fetch older messages I decided that when a new users signs into the app that I would activate a set of setup features along with the creation of the user account. This method would fetch all the messages associated with a user account and store them into another (encrypted) database and separate them into sent and received messages.
I studied facebook messengers database design (outlined on their engineering blog) and closely mirrored what they were doing. This would afford me different media types, addition of read receipts, typing indicators, group messages, soft and hard deletion of messages, and a lot more standard messenger type functionality. Since the app was now responsible for the storage and deletion of all messages on a users account this would require opening a stream to Twitters servers and listening for events that would modify the api. This would eventually prove to be catastrophic in terms of load on my servers as well as extremely expensive. I’m still not sure what the best development method I should have used to monitor all of these events at an economical level - however I created a little subsystem during user account creation to connect each user to their respective streams with Twitter and I also had to monitor whether these streams stayed connected or if I had to reconnect. This was accomplished through a CRON job that would check on a database of streams for connectivity and whether or not it was receiving data and heartbeats. This proved to be extremely reliable, effective, and simple.
The main idea behind version one of the app was to lay the groundwork for these features while simply doing what Twitter should have done - released their messaging product as a standalone app. So once my layer was hooked onto Twitters API and tested I moved on to the design and implementation of the user interface. I wanted to mimic the look and feel of iMessage - as they are the undisputed king of a great messaging experience - and that is exactly what I was able to accomplish. That is an astonishingly simple way of explaining the amount of work that went into this portion of the development - but the result was a smooth, intuitive, polished chat experience that looked and felt like it was produced by a team of engineers rather than a small company. There were a lot of aggravating caveats that I painstakingly had to find workaround for - for example the way the keyboard rendered itself, how to fetch data from the local store without hogging up all the processing power and lagging the screen rendering, etc.
After the user experience and interface design portion was done I began implementing the local store so that the app would work offline. I attempted to use a cross platform database - such as SQLite - but ended up using Core Data for its optimization and breadth of capabilities. This provided unique challenges as well and required significant work to create a smooth experience - and again took a considerable amount of time to implement correctly. I always knew we were going to expand the interface significantly and so the time investment spent on implementing things properly instead of as quick as possible payed measurable dividends on future iterations.
At this point all the pieces were in place for a version one release - and so thats what I did. On my birthday I made the app go live and submitted to Product Hunt and it quickly rose to the top spot that day. A few days later the app was featured on Techcrunch and the we began to see influencers start to use the app and within 2 months the user count had swelled to 80,000 installations, about half of which were active.
Oh hey, cool https://t.co/EBMlFyjymG— Anil Dash, America’s Most Beloved CEO (@anildash) November 20, 2015
Part 2: Porting to Android, Web and Increased Functionality
The feedback and press vindicated the idea and so there were a few obvious next steps. A messenger system is only as good as it’s ability to be able to contact whoever you want. This would mean creating an Android version and a Web version. In retrospect, working on 3 apps simultaneously while providing synchronous functionality was a big undertaking and would limit the polish and feel of the ecosystem - however we decided that in the circumstance of a messaging app that getting it in front of as many people was incredibly important at this stage of development.
Part 3: The Keyboard
The next version of the app was spent focussing on adding media attachment functionality. We implemented a sticker section, photo and video media uploading and offline viewing, voice note sending and receiving capabilities, and a location sharing feature. Each feature was implemented cross platform (including third party apps), and required a view controller to capture feature specific information (GPS coordinates, microphone activity) - which would then be encrypted and uploaded to the server - triggering a message event that would then need a rendered view on the other users device.
People loved sending stickers on Twitter from our platform. We also had worked on a Sticker store.
We used an ingenious workaround to upload photos and videos through Twitter's restricted API.
Voice Notes and Location Sharing showed the least engagement from our users, but allowed us to offer feature parity with some of the best apps on the market.
This rich media feature set gave us a leg up on any competition that was brewing and gave users cool and useful ways to interact on top of Twitter’s platform. This released cemented a great user experience and ultimately we became the best messenger app for Twitter.
Part 4: SMS, Voice, and Video
We were receiving good reviews on the app and had great momentum with regards to funding and hiring. We started running into some major issues with limits on the Twitter API. For each user we had to connect and process a user stream directly to Twitters servers. Each stream used up some of processing power - which is standard. The problem arose when we started getting more than 40, 000 downloads - this required a little bit of re-engineering of the load balancer to route streams across a fleet of servers hosted on AWS. This, in turn, started generating massive bills from AWS - in the range of $2000 - $4000 a month. We were basically paying a premium because of how Twitters API was set up - and to make matters worse we didn't even know if Twitter would approve our app to allow over 100, 000 users since any third party app with this amount of users would require a special whitelist. Since we were always cognizant of our control by Twitter we built everything in a way that would allow us to not rely on Twitter should we decide that was what we wanted to do, and after syncing a few months into the large bills we made the decision to pivot the messaging app.
Everyone is moving away from SMS and towards instant messaging, right? Wrong. SMS still makes up a majority of messages being sent and delivered around the world today - although it is slowly declining. People have many reasons to want to use sms to interact, and many reasons why they would want access to a second number (blocking robocalls, reservation numbers, business phone numbers, etc…). We were already modelled after iMessage, so new users would be able to understand what the app was trying to offer - and existing users might make the transition - although this was doubtful but worth a shot.
In tandem, we were building subsystems to support cross platform voice calls and video calls - which lends its potential nicely to phone numbers. This, along with the fact that were fully cross platform, gave us some good advantages in the market place - and so we began exploring API’s that would support our new direction. At first we took a look at completely in house solutions - but this industry proved very early and had an incredibly high time investment associated - so we decided to use the industry’s most well known and arguably best, Twilio for SMS.
We transitioned the database to be able to support SMS messages (by adding a new media transport field) - began integrating voice and video messaging cross platform using Google’s WEBRTC open standard - and launched the new app.
We had found a monetization strategy, had a lot of infrastructure behind us, and a great team working on the idea - however we had just about run out of money. Twitter’s API drove us to the ground, and we were having a hard time keeping Twilio bills payed while we iterated, researched, and developed the apps going forward. It was at this point that we realized that were were stretched very thin across our three apps and eventually made the decision to shut down the app. I still genuinely believe there is a market for Twitter Messenger, and also a market for an SMS/Voice messenger. There are apps that have since sprung up to offer these experience, but are plagued with bad design or poor focus.
It was a great 3 years.