We gave a talk at Drupalcon Portland 2022 about the Unified Government Website project we have been working on, as well as the migration of the Japanese Digital Agency website to a Headless CMS (Drupal + Next.js) as a feasibility study.
The talk consisted of three parts:
- Migration of the Digital Agency website to a headless CMS
- Research project for the realisation of a unified government website
- The future direction of the project
This blog post describes the first part above.
Overview of the project
The Digital Agency, established in September 2021, is running various projects with the goal of digitising Japanese society as a whole. Under the slogan of “Human-friendly digitalisation: No one left behind”, their aim is to design services from the user’s perspective.
Standardisation and unification of government websites (building a unified government website) is one of the projects underway to renew Japan’s information system.
Why a unified government website is required
“Research work pertains to verification of service infrastructure and design systems, etc., for the construction of a unified government website” points out the following issues regarding current ministry sites:
- Problems from the users’ perspective
- UI / UX inconsistency
- Similar content scattered across multiple sites
- Issues from the government perspective
- The sites of each ministry and agency are running on different systems, maintained and operated by different vendors, which increases the maintenance costs
For these reasons, the government has decided to aim for UI / UX improvement, as well as the unification of system maintenance/operation standardisation.
The migration of the Digital Agency website to a headless CMS was positioned as part of this project. It was a verification project to study the headless technology being considered for application to the unified government website and to apply the design system.
Migration of the Digital Agency website to a headless CMS
When the Digital Agency was launched in September 2021, the Agency’s website was created by a no-code website creation tool.
Headless architecture and rendering methods
The frontend of the new website is built with Next.js, while the backend is built with Drupal. Because the Static Site Generation (SSG) method was selected, the entire site is pre-built.
Drupal is used for the creation and management of the site’s content. The content is built by specific triggers, and the resulting HTML files are stored in a storage. These HTML files are distributed via a CDN.
Why Drupal was chosen
As of 2015, Drupal was beginning to be used as a headless CMS. At the time, the concept (and a feature) ‘entity’ was introduced to Drupal 7. With this change, Drupal has become a data store that handles pieces of data with various data types rather than handling data as a text for web content. With the introduction of entities, Drupal emerged as a general-purpose backend system that sets itself apart from typical web CMSs.
However, Drupal 7 was still built based on the assumption that requests are returned as rendered HTML pages. In response to the growing need for Drupal not only as a headless CMS but also as a backend for mobile apps and smart devices, Drupal 8 has undergone an architectural renewal from the API-first principle.
Through this complete rebuild, Drupal has transformed into a true and powerful headless CMS and has been a driver in leading many headless CMS projects to success.
With a track record and reliable community support, Drupal was chosen as the backend for the new Digital Agency website.
Why Next.js was chosen
Given the adoption of virtual DOM, the choices were Next.js and Nuxt.js. We chose Next for the following reasons:
- The Digital Agency site is considering adopting SSR in the future, but Next.js makes it relatively easy to switch between SSG and SSR at the setting level compared to Nuxt.js
- The availability of a development tool “Next-Drupal”, which can be used when using Next.js with Drupal, speeds up the development process
Content authoring using Markdown
Markdown is used for content authoring on the new Digital Agency websites. This improves the reusability of the site content once it is distributed through the API. Another reason to adopt Markdown is to adhere to one of Next’s best practices, which is to avoid including HTML in Next’s API responses.
Application of the design system
Prior to this verification project, a design system was created to improve the UI/UX of websites of ministries. The application of this design system to the new Digital Agency website was part of this validation project.
A description of the design system is found here (in Japanese only: https://digital-gov.note.jp/n/n78f6a2f82e48 ).
What we’ve learned - the technical aspect
About headless architecture
- More freedom for the frontend, as it is completely detached from the backend
- Accessibility tests can be performed at an early stage of development
The less good
- Increased development efforts, as two applications (i.e. the frontend and backend), had to be developed
- Diminishes Drupal’s advantages, as what Drupal generates (semi-) automatically cannot be used, such as menus and breadcrumbs
SSG is not suitable for the unified website
Current ministry websites often hold over 100,000 pages and the number only increases. Next.js rebuilds all the pages even if a single page is updated. Since building 100k pages requires a fair amount of time and server resources, adopting SSG was considered unrealistic.
With SSR, each page is rendered upon request; however, it will require significantly more resources than SSG.
JSON:API may not be the best choice
These points are related to bugs and the spec of JSON:API-related modules and may be fixed in the future.
- Open API for JSON:API, which we used to generate documentation, was not fully compliant with Open API Spec V3. Because of this, a large number of errors were seen when the documentation that was generated by the module was checked with a validator
- JSON:API Views did not return appropriate responses under certain conditions
Also, depending on the architecture you prefer / resource availability, Graph QL may be a better choice. With Graph QL, all the required page elements can be returned with a single request, whereas with JSON:API, a request needs to be sent for every page element, which increases the total number of requests sent to the backend.
This means (generally speaking) Graph QL would require more development efforts for the backend than the frontend, whereas with JSON:API, it is the other way around. Also, differences in the number of requests between the two approaches should be taken into account as you design the infrastructure.
There are no standard conventions for Markdown, and the expressibility of vanilla Markdown is limited. For example, you cannot express a table in vanilla Markdown.
Through our investigation, we have learned the pros and cons of various Markdown flavours and variants. For example, Github Markdown is specialised in describing issues, which makes it not entirely suitable for content authoring for websites.
We eventually discovered PHP Markdown Extras, which provided all the necessary elements for content authoring for the Digital Agency’s website. We used Markdown-it to allow the creation of tables, in-page links, description lists and footnotes.
What we’ve learned - the human/organisation aspect
Most of the human/organisation aspect of our learning converges to project management.
In a verification project such as this one, naturally, the final deliverables cannot be accurately defined at the start of the project. If detailed specifications are to be defined at the beginning of the project, the original purpose cannot be fulfilled. Therefore, the adoption of an agile project management method is a prerequisite for this type of project.
In order to adopt agile methods, however, the structure of the organisation must be ready to accommodate agile. For example, the decision-maker must be reachable when the team needs them.
In addition, each agile methodology defines its own processes, artefacts and terminologies. Therefore, an agreement must be made among the participants on which method to use and how to apply them, as well as to provide sufficient education for the participants to adhere to the rules defined by the method.
Through the migration of the Digital Agency’s website to a headless CMS, we gained a great deal of insight into headless technology and its application, as well as the application of the design system. At the same time, we were able to improve further the accessibility and functionality of the Agency’s website. The insights gained through this verification project will be utilised in future architecture selection and construction of the unified government website.
- Drupal 9 初心者講座について
- 第 1 回 歴史に見る Drupal の DNA
- 第 2 回 Drupal はフレームワークか？CMS か？他の CMS との比較
- 第 3 回 Drupal の特徴
- 第 4 回 Drupal 9 のインストール (1)
- 第 5 回 Drupal 9 のインストール (2)
- 第 6 回 Drupal にコンテンツを投稿してみる
- 第 7 回 Drupal のボキャブラリとタクソノミーの使い方
- 第 8 回 コンテンツ管理における Drupal と他の CMS との比較
- 第 9 回 Drupal のブロックシステム
- 第 10 回 Drupal の標準クエリビルダー Views の使い方
- 第 11 回 Drupal と他の CMS のクエリビルダー機能を比較
- 第 12 回 Drupal の多言語機能と他の CMS やサービスとの比較
- 第 13 回 Drupal の権限設定と WordPress や Movable Type との比較
- 第 14 回 Drupal のテーマシステムについて
- 第 15 回 Drupal の拡張モジュールの選定と使い方
- 第 16 回 Drupal をもっと知りたい方に向けた各種情報