The Instagram Ecosystem

Instagram is a well known microblogging and social networking service founded in October 2010; the users typically post images inhibiting a sense of privilege level where non-registered users can only see the posts by default and registered users can also like, comment and share these posts. Additionally, each user has to be created and authenticated by the service provider in order to use the service where profile customisation is an additional feature. In this regard, we consider the following requirements and limitations of Instagram below:

Requirements:

The following requirements have been considered:

• Retrieve and store images;
• "Following" users/pages;
• Publishing images/posts; and
• User notifications stemming from post activity such as likes and comments.

Limitations:

For the purpose of simplicity, the following limitations have been put in place:

• No instant messaging features;
• "Following" users/pages;
• No sharing of images from other pages; and
• Algorithms are not considered to provide users with the most interesting and relevant stories.

If we consider the flow of persistence here, then persistence is another way of expressing that our intention is to store some data for a later usage into another process. Let us view the entire relationship diagram below.

We’ve decided to opt for postgreSQL in this instance, simply because it is a standard relational database and we will store the information in multiple tables, some fields may overlap between tables. For instance userID may be found in the “like” and “comments” table to indicate which user the activity originated from. This way, updating one instance of userID automatically updates all relevant instances. An alternative would be a non-relational database, inter alias, a MongoDB which is great for systems that can scale well horizontally, however, updating entries that are duplicated may need numerous calls to the DB to update all instances of the same field. The entire relationship diagram may look like this:

• Likes: Each “like” would be associated to a type which would be a post or a comment as the user has an option to “like” either. Depending on the type, the TypeID in “likes” would be related to the UUID of the respective type. In addition, each “like” would be tied to a user by userID.
• Counter: This serves more as a “counter of likes”, every time a user likes a post the “likes” integer is incremented by one and reflected on the UI.
• Post/Comment: This feature type would contain the content, counter, timestamp and the user’s related activity.
• Binding: “Binding” is a very simple relation between a follower and a followee. Both fields would be in the form of their respective user IDs. Essentially, if we aim to query all the followers for a user, the query may look as follows:

SELECT User FROM Binding WHERE Followee = userID

Additionally, if we intend to query all the users one user follows, it may look like this:

SELECT Followee FROM Binding WHERE User = userID

Followers can also be represented graphically, e.g., where orange single directional arrows indicate a “following” and green bi-directional arrows indicate mutual “following”:

Further on, we’ve opted for a Microservice architecture as opposed to a monolith because with such a large system and much room for improvement we thought it was best to separate the duties into smaller dedicated services:

The general scenario is as follows:

• User sends a POST HTTP request along with their userID, requesting a “feed” (i.e., top 20 most recent images from their followers);
• The gateway receives this request and forwards it to a relevant service, in our case the “feed service”. Additionally, the gateway can also take care of the user’s authentication;
• “Feed service” queries the “follow service” to retrieve all the followers;
• “Feed service” queries the “post service” to retrieve the most recent post from each followee, limiting the number to 20 posts to reduce the overall network load (i.e., load on the database and load on the service); and
• This response is sent back to the user.

Design Decisions:

• Each Microservice is scalable and involves multiple instances to produce a more resilient system. If we only had one instance and it’d fail, then we’d have a single point of failure and the users would not be able to request their feed until the service comes back online. Having multiple replicas implies that if one replica goes offline, the incoming requests can be forwarded to the other replicas via a load balancer.
• A gateway contains a system snapshot created by the load balancer who coordinates with all the services within the cluster. It is expensive to keep querying the load balancer for each request, instead the load balancer creates a snapshot and updates it periodically in order for the gateway to use it to forward user requests to relevant services for handling. For instance, a get feed request should be forwarded to the feed service and get followers request should be forwarded to the follow service.
• Pre-compute user feed as recomputing all the posts on the feed is too expensive to conduct this per request. Instead, every time a user creates a post, this action creates a notification request to a post service:
  • The post service updates the feed service on this event; and
  • The post gets stored in the feed’s data store for all the relevant user’s queue, therefore, every time the user requests their feed, no computation is necessary. The interactions outlined in blue may look as follows:

To conclude, we had the pleasure of exploring the Net Reply’s take on in terms of designing an Instagram-like ecosystem. Since Instagram is an extremely complex system, we introduced some limitations in order to cater it for the wider audience and keep this blog simple yet concise while still conveying the main system design concepts. Feel free to reach out with any questions/feedback here.