Implementing GraphQL for Backlog’s Boards feature

GraphQL — the query language made by Facebook — has gained a lot of popularity in recent years and for good reasons. The biggest selling point is that it allows frontend developers to precisely select the data they need in the UI, which can substantially reduce the size of response payloads. Another significant advantage is that backend and frontend share a schema which exposes what queries and mutations are available. This schema can be used to generate types for Typescript, which in turn acts directly as documentation for developers and as an entry point to explore the API.

At Nulab, we decided to implement GraphQL for our project management app Backlog, specifically for querying data for cards on our Kanban-style Boards feature. In this article, I’ll discuss how we did this.

Implementing and integrating the GraphQL server with Backlog

Our development team at Nulab had shown interest in GraphQL in the past and had already experimented with the query language on some internal projects. These experiments convinced us that GraphQL brought a lot of advantages over traditional HTTP API. We were in love with the type generation between frontend and backend, how easy it was to write GraphQL queries to select data, and all the tools the GraphQL community has developed. When we decided to develop the Kanban feature in Backlog, we knew we wanted to use GraphQL and leverage all of its benefits in our development process.

At the beginning of the project, we identified three options to integrate a GraphQL server:

1. Use RPC to communicate with the Backlog server
2. Use parts of the Backlog backend code as a library
3. Don’t use GraphQL and create a new endpoint in Backlog instead

Option 1

The first option was to create a clear separation from the Backlog server and only send some calls through RPC to do the business logic. It sounded nice on paper, but it also added a point of failure since network communication can fail. Because our delivery time was short, we didn’t want to deal with inter-server communication failures and all the tools we would need to monitor and recover from such errors.

Option 2

The second option was appealing because Backlog follows an onion architecture. Since we were already using Scala for Backlog’s backend, we could reuse the code as a library and call the application service we needed directly from the GraphQL server. This option had less risk than the previous one. And fortunately, there is an excellent library in Scala on how to write a GraphQL server.

Option 3

The last option was the safest choice, which was simply not to use GraphQL. But we’d have to go back to an HTTP API, which is okay, but we wouldn’t be able to try GraphQL in Backlog.

Solution

In the end, we decided to implement a GraphQL API on another server other than Backlog. We also decided to write the Kanban feature in a purely functional style from backend to frontend, so we could control side-effects while writing expressive, composable, and testable code.

We put a lot of emphasis on type safety in our solution to reduce the risk of bugs. By leveraging the power of types, we were able to model our domain and all the Kanban-related operations. As a result, the compiler had more information to help us and could eliminate a class of bugs we otherwise would not have been able to catch. We also wrote a lot of tests to prevent regression or bugs from happening. Both the compiler and tests were essential for tackling the robustness of the code while giving us a high level of confidence in the result.

The Boards feature architecture

The architecture for our Kanban-style Boards feature at this point was as follows:

Backlog and Kanban are on different servers, but they share the same database and the same business code. This has some consequences in terms of configuration, as the Kanban server must include Backlog configuration to work. Keeping the settings in sync is the price to pay for reusing Backlog as a library instead of using RPC.

For frontend developers, GraphQL provides a set of tools they can use to discover the API like GraphiQL and Apollo client. Apollo directly supports writing GraphQL queries in your code with auto-completion and checking your query against the GraphQL schema.

const cardFragment = gql`
  fragment CardContent on KanbanCard {
    id
    issue {
      id
      issueType {
        id
        color
        name
      }
      summary
      issueKey
      projectId
      assignee {
        id
        name
        userId
        icon
      }
      resolution
      priority
      status {
        id
        name
        color
        isCustomStatus
      }
      estimatedHours
      actualHours
      dueDate
      parentId
      created
      updated
    }
    order
  }
`

const listQuery = gql`
  query getCards($cardSearch: CardSearchInput!) {
    cards(cardSearch: $cardSearch) {
      total
      cards {
        ...CardContent
      }
    }
  }
  ${cardFragment}
`

As a result, we were able to provide an API that is easily discoverable by frontend developers, typed, and that allowed developers to select the data the UI needs.

Making Boards work in real-time

With only the GraphQL query, our Boards solution would not update cards in real-time. We needed to find a solution to send a notification to the client about all card-related events.

There were four types of events in Kanban we were interested in listening to:

• Someone updated a card
• Someone moved a card
• Someone added a card
• Someone deleted a card

For example:

We want the “User 2” screen to update if “User 1” is updating the board. For that, we needed to send a notification to “User 1” and make the application refresh the cards.

We also needed to keep in mind that the issue can be updated on other screens or by calling the Backlog API. This added complexity in terms of deciding who should be responsible for sending notifications, as the number of events we had to listen to was larger than before. The activities to listen to in the Backlog API and Backlog web server are:

• Someone has updated an issue.
• Someone has deleted an issue.
• Someone has created an issue.

We translated most of these events into Board events. We added a Redis notification queue, to which all Kanban events would be pushed, and a WebSocket server to listen to the Redis queue and dispatch these events further to the clients.

Now our architecture looks like this:

We tried implementing the WebSocket server in Go at first. But due to some technical issues in AWS load balancing in some spaces, we would have had to fall back to HTTP polling in some cases. Since our Go notification server doesn’t support HTTP polling, we decided to switch to Node.js and socket.io, which solved the issue.

Conclusion

Designing and implementing Backlog’s first use of GraphQL was very interesting. We learned a lot, and we were able to deliver a quality Kanban-style application in Backlog that users are enjoying.

We think GraphQL has a lot of potential for Backlog in the future, beyond Boards. And there are still a lot of things we want to improve in the future for Boards, like simplify the configuration, for example. While this article focused mostly on Backend and GraphQL, there is certainly a lot to talk about in your Frontend architecture too — but we will save that for another blog post.