System Design: Build URL Shortening Service using Ktor (Part-1)

System Design: Build URL Shortening Service using Ktor (Part-1)

In this blog, we are going to learn about the most common question asked in System design rounds. We will also implement the same using Ktor.

Hey everybody,

While we prepare for interviews for our next Software jobs for SDE-2+ levels we generally asked questions about System Designing and one of the most common questions we come across is,

Build URL Shortening Service

In this blog series, we are going to learn,

  • Idea we are going to solve.
  • Things to take care of.
  • Setting up the project.
  • Setting up Database Layer
  • Moving towards building feature.
  • And many more.

Let us start without any delay.

Idea we are going to solve.

First, we have to discuss the idea itself. Think of this as your own personal Bit.ly which you can use to build your own URL service. But, here we need to chalk down the idea itself what we need which building this.

This will be useful even for your system design interview.

  • We need to build a route to take a long original URL and return a short URL.
  • Short URL should not be very easily predictable by the end-user.
  • Now, we need another route to handle the short URL which will be mapped to the original URL.
  • It should have an expiration time so that the short URL is inactive after a certain duration (out of scope for this blog series).

This to take care of.

To build up this solution from the ground up, we need to pre-plan for it. We need a Database service to handle the creation and mapping of the shortened URLs. We also need a definite architecture to make a robust app as well.

Our architecture will look like,

arch-design-url-short.jpg

Setting Up the Project.

We will be creating the project from Intellij IDE itself using the Ktor plugin. If you want to learn more about how to create a project, check the following blog.

In this project, the plugin which we are going to use is

  • ContentNegotation
  • Gson
  • Status Page
  • Routing
  • Location

Screenshot 2021-07-04 at 5.41.23 PM.png

Now, create the packages like this. Where,

  • base will hold all providers, Routing, Status and Serialization
  • di will hold all the manual locators we have created for Domain, Database etc.
  • feature will have again a package of url which will contain sub-packages like repository, domain, service, etc.
  • util will have all your utility functions like BaseResponse, etc.
  • Lastly, Application will have the module extensions and the registration of the features.

Setting up database layer

In this project as well, you will use Kmongo, a kotlin wrapper over the official Mongo java client. You can read more about it here.

Implement the coroutine version of the library like,

implementation("org.litote.kmongo:kmongo-coroutine:4.2.8")

Now, create a package in,

base -> providers -> database

Here, first, create the DatabaseProvider interface and add the following functions,

interface DatabaseProvider {

    val initializeName: String

    val mongoClient: CoroutineClient

    val database: CoroutineDatabase
}

Now, create another class that will work as the implemented version of the above interface and it override. It will look like,

class DatabaseProviderImpl(private val clientName: String) : DatabaseProvider {

    override val initializeName: String
        get() = clientName

    override val mongoClient: CoroutineClient
        get() =


    override val database: CoroutineDatabase
        get() =

}

You can see here, we are passing the clientName from the constructor because we want to follow the dependency principle of passing it from outside. Here, clientName will be used as the name for my Mongo client.

Now, let us complete the code. The final code will look like this,

class DatabaseProviderImpl(private val clientName: String) : DatabaseProvider {

    override val initializeName: String
        get() = clientName

    override val mongoClient: CoroutineClient
        get() = KMongo.createClient().coroutine

    override val database: CoroutineDatabase
        get() = mongoClient.getDatabase(initializeName)

}

Lastly, we need provide the DatabaseProvider to other feature services we need to setup its locator in,

di -> DatabaseLocator

and it will look like,

object DatabaseLocator {

    private fun provideClientName(): String {
        return "ktor-url-shortner"
    }

    fun provideDatabaseProvider(): DatabaseProvider {
        return DatabaseProviderImpl(provideClientName())
    }
}

Here, if you look closely we first created a client name using the provideClientName() function which you will use to instantiate the DatabaseProviderImpl class of the type DatabaseProvider.

We use the return type DatabaseProvider and not DatabaseProviderImpl because as a rule of abstraction, we don't expose the implementation of the function but only the methods/functions.

The database setup is complete.

Moving towards building feature.

As a first step, we will create a package called url in the feature module which will have sub-packages namely, domain, request, routing, service, repository, and entity.

Step 01.

First, let us create an entity inside the entity package that will be responsible for the collection structure in our database. We will name it, UrlEntity. It looks like,

data class UrlEntity(
    @BsonId
    val urlId: String = ObjectId().toString(),
    val originalUrl: String,
    val shortUrl: String,
    val createdAt: String,
    val urlHitCount: Int = 0
)

Here,

  • urlId is the BsonId. Read more about it here.
  • originalUrl will be the long URL that we have to convert.
  • shortUrl is the converted URL.
  • createdAt is the time at which the URL was converted
  • urlHitCount represents the number of times we have called the short URL.

Since, you have created the entity structure, let us quickly set up the Collection in DatabaseProvider by adding,

val urlCollection: CoroutineCollection<UrlEntity>

and its implementation in DatabaseProviderImpl will look like,

override val urlCollection: CoroutineCollection<UrlEntity>
        get() = database.getCollection()

Your structure of Collection is not setup using the UrlEntity that we created.

Step 02.

Now, you will setup your UrlService interface and its implementation class called UrlServiceImpl. This class will be responsible for doing all your database operations. UrlService interface looks like,

interface UrlService {

    suspend fun createShortUrl(url: String): String?

    suspend fun findShortUrl(url: String): String?

    suspend fun findOriginalUrl(url: String): String?

    suspend fun checkIfUrlIsPresent(url: String): Boolean

    suspend fun getTotalCount(url: String): Int?
}

Here,

  • createShortUrl is responsible for creating a short URL based on a given URL.
  • findShortUrl is responsible for getting the short URL based on a given long URL.
  • findOriginalUrl is responsible for getting the original URL based on a given short URL.
  • checkIfUrlIsPresent is responsible for checking if the original URL is present or not.
  • getTotalCount will return the number of times the short URL was pinged.

The UrlServiceImpl class will look like,

class UrlServiceImpl(private val urlCollection: CoroutineCollection<UrlEntity>) : UrlService {

    override suspend fun createShortUrl(url: String): String? {

    }

    override suspend fun findShortUrl(url: String): String? {
    }

    override suspend fun findOriginalUrl(url: String): String? {
    }

    override suspend fun checkIfUrlIsPresent(url: String): Boolean {
    }

    override suspend fun getTotalCount(url: String): Int? {
    }
}

Here, you can see we are again passing the CoroutineCollection from outside via the constructor.

So, for that first, you need to create the ServiceProvider in base and ServiceLocator in di package. Your ServiceProvider looks like,

interface ServiceProvider {

    fun provideUrlService(): UrlService
}

and its implementation class looks like,

class ServiceProviderImpl(private val databaseLocator: DatabaseLocator) : ServiceProvider {

    override fun provideUrlService(): UrlService {
        return UrlServiceImpl(databaseLocator.provideDatabaseProvider().urlCollection)
    }
}

Here, DatabaseLocator is used to provide the urlCollection to the ServiceImpl class.

Now, let us update the ServiceLocator which will work as our DI. It will look like,

object ServiceLocator {

    fun provideUrlService(): UrlService {
        return provideServiceProvider().provideUrlService()
    }

    private fun provideServiceProvider(): ServiceProvider {
        return ServiceProviderImpl(DatabaseLocator)
    }
}

Finally, we are done setting up the providers and the locator for UrlService.

Let us finally update the UrlServiceImpl like,

class UrlServiceImpl(private val urlCollection: CoroutineCollection<UrlEntity>) : UrlService {

    override suspend fun createShortUrl(url: String): String? {
        val shortUrl = generateRandomUrl()
        val request = UrlEntity(
            originalUrl = url,
            shortUrl = shortUrl,
            createdAt = Date().toInstant().toString(),
        )
        val isShortUrlCreated = urlCollection.insertOne(request).wasAcknowledged()
        return if (isShortUrlCreated) {
            shortUrl
        } else {
            null
        }
    }

    override suspend fun findShortUrl(url: String): String? {
        val urlEntity = urlCollection.findOne(UrlEntity::originalUrl eq url)
        return urlEntity?.shortUrl
    }

    override suspend fun findOriginalUrl(url: String): String? {
        val urlEntity = urlCollection.findOne(UrlEntity::shortUrl eq url)
        updateCount(urlEntity)
        return urlEntity?.originalUrl
    }

    override suspend fun checkIfUrlIsPresent(url: String): Boolean {
        return findShortUrl(url) != null
    }

    override suspend fun getTotalCount(url: String): Int? {
        val urlEntity = urlCollection.findOne(UrlEntity::shortUrl eq url)
        return urlEntity?.urlHitCount
    }

    private suspend fun updateCount(urlEntity: UrlEntity?) {
        urlEntity?.let { entity ->
            val count = entity.urlHitCount.plus(1)
            val newEntity = entity.copy(urlHitCount = count)
            urlCollection.updateOne(UrlEntity::shortUrl eq entity.shortUrl, newEntity)
        }
    }
}

Here, in the createShortUrl function you can see it has a generateRandomUrl function which is present in,

util-> Util.kt

and it looks like,

fun generateRandomUrl(length: Int = 6): String {
    val allowedChars = ('A'..'Z') + ('a'..'z') + ('0'..'9')
    return (1..length)
        .map { allowedChars.random() }
        .joinToString("")
}

The above block of code will generate a random alpha-numeric string of length 6 which will represent the endpoint of the shortened URL.

Also, in the findOriginalUrl function we check if the URL is present or not. If we get the URL then we call another function there called updateCount() which is responsible for increasing the urlHitCount by 1 every time the function is called. This basically represents the number of hits the short URL has been pinged.

Rest all the functions in the UrlServiceImpl is self-explanatory as they are mere MongoDB operations. If you want to read more about the mongo operations check this blog.

That is all for Part 1 of this blog. Click here for Part -2

If you like the series, do share it with people :)

Check the code here.

Thank you for reading :) Hope you learned something from my experience.

If you have anything more to discuss, I will be happy to get connected at hi_man_shoe.

See you in the next blog :)

Did you find this article valuable?

Support Himanshu Singh by becoming a sponsor. Any amount is appreciated!