Drupal portfolio : e-Stat: Japanese Government Statistics Portal

e-Stat 政府統計の総合窓口

The e-Stat government statistics portal is a one-stop data portal, providing the public with Japanese government statistical data from all government ministries. The primary focus of the site is to allow users to search and find the data they need, but it also provides various other functionality relevant to users of the site.

Household, population, and other statistics collected as part of the national census that takes place every five years is provided to the public on this portal. These statistics are used to aid in marketing, consulting, research, policy planning and a variety of other purposes.

Site Rebuild Goals

The original site, built 10 years previous, had a massive back-end built in the JAVA programming language. The site design and layout, known as the front-end, and the data processing and site administration, known as the back-end, were not separated from each other, therefore requiring specialist programmers to add and manage site contents. This made for expensive running costs and an inability for site administrators to directly interact with the system. As such, the new system needed to maintain the massive existing data back-end, while allowing the client the ability to directly manage their contents, reducing running costs.

ANNAI's Solution

We proposed building a Drupal front-end, where Drupal pulls data from the existing Java back-end, allowing for content management within Drupal.

Why we chose Drupal

Drupal has a proven track record world-wide as a data portal in the statistics field. The Drupal framework allows for creating a set of modules, each interacting with a specific REST API, to pull data from remote systems and display it in blocks.

The majority of Drupal installations use a MySQL database for data storage, however the e-Stat system would instead use the existing Java back-end for data storage. This would allow for site administrators to manage contents through Drupal, and allow content to be displayed through Drupal's TWIG template engine. TWIG lets developers write HTML in a secure manner in which they cannot accidentally expose sensitive data. This means non-specialists can work on the site, reducing costs of design changes from those of the original site.

Drupal also has a thriving open-source community built around it, with tens of thousands of community sponsored modules that can be used to add various functionality to Drupal sites. For situations where community modules do not satisfy the system requirements, Drupal has an advanced framework that we at ANNAI use to customize systems as necessary.

Technical Overview

The existing Java back-end would manage the data storage, and Drupal would manage contents and display.

estat design pattern

Drupal - Java Integration Overview

This configuration allows for statistics to be handled by the Java back-end, while allowing for administrators to use Drupal to create and edit pages through the admin interface, without requiring them to contact the vendor. This allows for real-time management of the site, while at the same time reducing system running costs.

Using Drupal to implement a system allowing for content creation and moderation by multiple simultaneous users

The e-State renewal project was a collaboration between ANNAI, who created the Drupal front-end to manage contents, and a third party, referred to here as Company A, who were responsible for maintaining the Java back-end, and the REST API used to serve Java data to Drupal.

ANNAI Responsibilities Company A
  • Site build (using Drupal)
  • Site design/UI design
  • Overall system architecture
  • Drupal-Java integration architecture
  • Technical guidelines/Sample implementation
  • Java Back-end system maintenance
  • Implementation of search-engine, to retrieve data from Java
  • Creation and maintenance of REST API and Drupal modules used to serve search result data to Drupal

By working with the existing Java back-end, the site renewal was able to be implemented within a tight development deadline by using Drupal's module system to create integration between Drupal and the back-end.

Company A created Drupal modules that interacted with the back-end REST API. In this way, back-end system maintenance and front-end Drupal development could be kept entirely separated, allowing for a clear delineation between responsibilities.

Issues During Development

It requires more effort and resources to upgrade an existing system, than it does to build a new one

While creating a new system does require both effort and resources, upgrading existing systems take more, as there are existing requirements that must be maintained through the upgrade. In the initial stages of the upgrade proposal, a migration of data from the existing back-end into Drupal was considered, keeping everything within a single system. However, we determined that going this route introduced a considerable amount of risk, as it meant both building a new front-end and a new back-end. By only changing the front-end, we could ensure the project was finished within the tight deadline.

Data display system provided by the back-end system

Data provided by the back-end

The result was a system in which the back-end handles data and business logic, while Drupal provides data-output and content management functionality.

This architecture allows for maintaining the legacy data system, while simultaneously implementing a modern new design/user experience, and allowing for site owners to administer content. And from the developer's viewpoint, using Drupal allowed for leveraging Drupal's contributed module ecosystem to provide powerful functionality at a reduced cost.

ANNAI provides total support from proposal to design and user experience, through training on system usage

ANNAI's strength lies in our ability to use Drupal to its strengths, allowing us to propose effective development proposals, consulting, design and user experience and system training.

As seen in our renewal of the e-Stat system, we are able to take multiple systems and link them together with Drupal, by clearly identifying the division of duties between teams. This allowed us to provide consulting and support to implement a system where multiple teams were able to develop the project.

For this project, we started with doing a proof-of-concept to show integration between Drupal and a separate back-end. This showed our company's ability to not just put together a proposal, but also show implementation-level consulting.

For your complete expert Drupal needs, we are ANNAI. Our clients' success is our success.