Reverse Engineering APIs

Overview

When generating a manifest file for your web proof, you will require a data source. This could be as simple as finding a public API, however in most cases, it will be more complicated than this. You may want data that is not publicly available. You may want data that requires user authentication. This kind of data is undocumented since it is not meant to be consumed by public developers, rather it is meant for the website’s development team.

This guide hopes to assist in shedding some light on some tools to assist you in your journey in acquiring the information needed to construct your manifest and prepare.js files.

While there is no step-by-step guide on how to get the information needed for your data, there are usually a couple preliminary steps.

Prepare Url

The first preliminary step is to identify a proper url for users to sign into (or for public APIs to call, but for our use, a login url). The most important thing about this URL, is it should not only be accessible by a mobile device, but it should not redirect to a mobile app, or app store, upon login. This is vital, because when the user is in the Attest Mobile app, we don’t want them redirected to a different app upon authenticating.

Selecting Data

The second preliminary step is to identify your target data. What specific data are you trying to acquire? A number of twitter followers for a given user? A purchase from a website at specific data, or range of dates? Maybe you want to see if a user has at least a certain balance in their account. If you are unsure of your target data, but have a target site, that is also okay, you can browse the data to see what is useful to extract.

At a minimum, we are looking for one main piece of data, an authentication token. In many cases, we may also be looking for more data, such as user data such as an id, to construct an api url, information to construct a payload to pass along, or additional required security information such as a CSRF token.

Tools

Once you have an idea of the data you are looking for and the url the end-user uses to authenticate with, it is time to enter detective mode. For this, there are two tools that will become your best friends: The Chrome DevTools and an API Testing Tool such as Postman. Throughout the process, whenever you find something that works, make sure you document it, so you don’t lose that information.

Authentication Tokens

When looking for an authentication token, the first step is to open your Chrome DevTools, go to the Network Tab, find an authenticated API, and see how it is passing the authentication information along. From there, it can help you determine the source of the authentication. More than likely, it came from an authentication cookie. You can view cookies within the Chrome DevTools under the Application->Cookies tab. From there, you can see a list of cookies associated with the domain you are on. An authentication cookie will usually (but not always) be set as both Secure and HttpOnly. If you are having issues finding which cookie may be the auth cookie, a good (but slower) way to find it, is through brute force. Starting from the top, delete a single cookie, and refresh the web page. Did you get logged out? If not, delete the next cookie. Repeat this process until you are logged out. Whichever cookie is the one that logs you out is the auth cookie.

This is not to say that the auth cookie is the only important cookie, or piece of data, required for the api. It all depends on the implementation and security requirements of the website, and even the experience and preferences of the developer. Other sources of authentication data can include Local Storage and even the DOM (Spotify is a culprit of this, they receive the access token through a script and inject it into their DOM).

Once you have your auth cookie, and have noted how to pass it along in the API request, it is time to move onto getting any other data you need. This data can be any data needed to construct a dynamic url (e.g. /users/{user_id}/profile), or maybe a body for a GraphQL query. The process for this is a bit different than the cookie flow, and heavily depends on how the site has implemented their APIs. As you go through this process, take note of how you found this, as you will be writing a javascript script to extract this code in a later step.

Data APIs

If a site has implemented a straight forward GraphQL Api, it may be simple to piece together the schema from looking though existing APIs, or if there is an existing API, you may be able to find the data you need within the response. As before, take note of how you found this, as you will be implementing a script later.

Testing Data APIs

This is where a tool, such as Postman will come in handy. Since you have your endpoint, and your auth token, open up Postman, and see if you are able to successfully replicate the API call you want to make. Adjust any parameters as necessary. You may need additional headers, such as a CSRF token, or more standard ones like Content-Type and User-Agent. In Practice, Pluto handles the basic headers for you, so you won’t need to worry about these, but for Postman or other tools, you may need to set them manually.

Next Steps

Once you are able to recreate your API successfully in your tool, it’s time to move on to the Manifest Builder, which will guide you through actually generating the manifest, and building the prepare.js with your newfound information.