ANNAI magazine
Drupalcon 2022 logo
Table of content

We gave a talk at Drupalcon Portland 2022 about the Unified Government Website project we have been working on, as well as the migration of the Japanese Digital Agency website to a Headless CMS (Drupal + Next.js) as a feasibility study.

The talk consisted of three parts:

  1. Migration of the Digital Agency website to a headless CMS
  2. Research project for the realisation of a unified government website
  3. The future direction of the project

This blog post describes the first part above.

Overview of the project

The Digital Agency, established in September 2021, is running various projects with the goal of digitising Japanese society as a whole. Under the slogan of “Human-friendly digitalisation: No one left behind”, their aim is to design services from the user’s perspective.

Standardisation and unification of government websites (building a unified government website) is one of the projects underway to renew Japan’s information system.

Why a unified government website is required

Research work pertains to verification of service infrastructure and design systems, etc., for the construction of a unified government website” points out the following issues regarding current ministry sites:

  • Problems from the users’ perspective
    • UI / UX inconsistency
    • Similar content scattered across multiple sites
  • Issues from the government perspective
    • The sites of each ministry and agency are running on different systems, maintained and operated by different vendors, which increases the maintenance costs

For these reasons, the government has decided to aim for UI / UX improvement, as well as the unification of system maintenance/operation standardisation.

The migration of the Digital Agency website to a headless CMS was positioned as part of this project. It was a verification project to study the headless technology being considered for application to the unified government website and to apply the design system.

Migration of the Digital Agency website to a headless CMS

When the Digital Agency was launched in September 2021, the Agency’s website was created by a no-code website creation tool.

Headless architecture and rendering methods

The frontend of the new website is built with Next.js, while the backend is built with Drupal. Because the Static Site Generation (SSG) method was selected, the entire site is pre-built.

An architecture diagram of the digital agency website

Drupal is used for the creation and management of the site’s content. The content is built by specific triggers, and the resulting HTML files are stored in a storage. These HTML files are distributed via a CDN.

In terms of the rendering methods, there are several options: Single Page Application (SPA) updates elements of a page without requiring reloading of an entire page; Server Side Rendering (SSR) dynamically builds a page upon receiving a request. SSG was selected because 1) the Agency aimed to serve pages/content that didn’t require JavaScript to give maximum consideration to accessibility, and 2) the Agency was interested in using SSG for improved security and performance.

Why Drupal was chosen

As of 2015, Drupal was beginning to be used as a headless CMS. At the time, the concept (and a feature) ‘entity’ was introduced to Drupal 7. With this change, Drupal has become a data store that handles pieces of data with various data types rather than handling data as a text for web content. With the introduction of entities, Drupal emerged as a general-purpose backend system that sets itself apart from typical web CMSs.

However, Drupal 7 was still built based on the assumption that requests are returned as rendered HTML pages. In response to the growing need for Drupal not only as a headless CMS but also as a backend for mobile apps and smart devices, Drupal 8 has undergone an architectural renewal from the API-first principle.

Through this complete rebuild, Drupal has transformed into a true and powerful headless CMS and has been a driver in leading many headless CMS projects to success.

With a track record and reliable community support, Drupal was chosen as the backend for the new Digital Agency website.

Why Next.js was chosen

Given the adoption of virtual DOM, the choices were Next.js and Nuxt.js. We chose Next for the following reasons:

  • The Digital Agency site is considering adopting SSR in the future, but Next.js makes it relatively easy to switch between SSG and SSR at the setting level compared to Nuxt.js
  • The availability of a development tool “Next-Drupal”, which can be used when using Next.js with Drupal, speeds up the development process

Content authoring using Markdown

Markdown is used for content authoring on the new Digital Agency websites. This improves the reusability of the site content once it is distributed through the API. Another reason to adopt Markdown is to adhere to one of Next’s best practices, which is to avoid including HTML in Next’s API responses.

Application of the design system

Prior to this verification project, a design system was created to improve the UI/UX of websites of ministries. The application of this design system to the new Digital Agency website was part of this validation project.

A description of the design system is found here (in Japanese only: https://digital-gov.note.jp/n/n78f6a2f82e48 ).

What we’ve learned - the technical aspect

About headless architecture

The good

  • More freedom for the frontend, as it is completely detached from the backend
  • Accessibility tests can be performed at an early stage of development

The less good

  • Increased development efforts, as two applications (i.e. the frontend and backend), had to be developed
  • Diminishes Drupal’s advantages, as what Drupal generates (semi-) automatically cannot be used, such as menus and breadcrumbs

SSG is not suitable for the unified website

Current ministry websites often hold over 100,000 pages and the number only increases. Next.js rebuilds all the pages even if a single page is updated. Since building 100k pages requires a fair amount of time and server resources, adopting SSG was considered unrealistic.

With SSR, each page is rendered upon request; however, it will require significantly more resources than SSG. 

JSON:API may not be the best choice

These points are related to bugs and the spec of JSON:API-related modules and may be fixed in the future.

  • Open API for JSON:API, which we used to generate documentation, was not fully compliant with Open API Spec V3. Because of this, a large number of errors were seen when the documentation that was generated by the module was checked with a validator
  • JSON:API Views did not return appropriate responses under certain conditions

Also, depending on the architecture you prefer / resource availability, Graph QL may be a better choice. With Graph QL, all the required page elements can be returned with a single request, whereas with JSON:API, a request needs to be sent for every page element, which increases the total number of requests sent to the backend.

This means (generally speaking) Graph QL would require more development efforts for the backend than the frontend, whereas with JSON:API, it is the other way around. Also, differences in the number of requests between the two approaches should be taken into account as you design the infrastructure.

Markdown

There are no standard conventions for Markdown, and the expressibility of vanilla Markdown is limited. For example, you cannot express a table in vanilla Markdown.

Through our investigation, we have learned the pros and cons of various Markdown flavours and variants. For example, Github Markdown is specialised in describing issues, which makes it not entirely suitable for content authoring for websites.

We eventually discovered PHP Markdown Extras, which provided all the necessary elements for content authoring for the Digital Agency’s website. We used Markdown-it to allow the creation of tables, in-page links, description lists and footnotes.

What we’ve learned - the human/organisation aspect

Most of the human/organisation aspect of our learning converges to project management.

In a verification project such as this one, naturally, the final deliverables cannot be accurately defined at the start of the project. If detailed specifications are to be defined at the beginning of the project, the original purpose cannot be fulfilled. Therefore, the adoption of an agile project management method is a prerequisite for this type of project.

In order to adopt agile methods, however, the structure of the organisation must be ready to accommodate agile. For example, the decision-maker must be reachable when the team needs them.

In addition, each agile methodology defines its own processes, artefacts and terminologies. Therefore, an agreement must be made among the participants on which method to use and how to apply them, as well as to provide sufficient education for the participants to adhere to the rules defined by the method.

Summary

Through the migration of the Digital Agency’s website to a headless CMS, we gained a great deal of insight into headless technology and its application, as well as the application of the design system. At the same time, we were able to improve further the accessibility and functionality of the Agency’s website. The insights gained through this verification project will be utilised in future architecture selection and construction of the unified government website.

 
フッターの採用情報
 
Mori Sugimotoの写真

The author of this post: Mori Sugimoto

Since 2006, Mori has been involved in a number of large-scale Drupal projects in the US/Europe as a developer, architect and consultant.

He has been actively making contributions to the Drupal community by:

  • Translating Drupal core into Japanese
  • Organising Drupal events and meetups
  • Volunteering and presenting at Drupalcons
  • Taking a coordinating role in the Drupal Security Team

Mori is based in Utrecht, the Netherlands.