Smart Manufacturing / MESETAO Tech2024-09-20

ETAO Tech: Three Databases Collapsed Back Into One

To absorb high-frequency acquisition, deep traceability queries and semi-structured intermediate results at once, the MES had bolted InfluxDB and MongoDB onto MySQL — pushing work that belongs in a database up into the application. Migrating to MatrixOne returned it to a single database, in under a week.

ETAO Tech

Founded in 2012, ETAO Tech is a software and IT services company focused on digital transformation for electronics manufacturing, bringing software systems, data platforms and AI into the smart factory. It serves over a hundred digital factories in the sector, with more than 15,000 connected IoT endpoints.

3 → 1
Database technologies
< 1 week
Full migration
15,000+
Connected IoT endpoints
TB / month
Acquisition growth per plant

The challenge

ETAO builds its own MES for the manufacturing execution layer, covering data management, planning and scheduling, production dispatch, inventory, supply chain, equipment and quality management — a full manufacturing collaboration platform, already delivered to over a hundred customers. Alongside the software it supplies the whole equipment data acquisition stack: reading directly from device PLCs, through external edge collectors, or from console hosts.

The first pressure was volume. Manufacturers increasingly expect their MES vendor to retain and use everything collected, and a typical electronics plant with dozens of production lines accumulates terabytes within a month of bringing MES and acquisition online. To absorb those writes, InfluxDB was added alongside MySQL.

The second was query. Report analysis and product traceability — and further out, predictive maintenance and process optimization — all carry heavy computation, and traceability queries reaching back months or years were well beyond MySQL. Engineers moved to pre-computing in the application layer, which produced large volumes of semi-structured JSON, so MongoDB was added to hold it. That helped, but MongoDB offers no SQL for multi-dimensional analysis, so the pre-computation logic stayed in the application.

The third pressure was the sum of the first two. Data sat across three databases — people, materials, process and equipment in MySQL, acquisition data in InfluxDB, intermediate results in MongoDB. Serving a business application meant repeatedly fetching, computing and reconciling across all three, and the business logic became correspondingly complex.

A great deal of what a database should do had been pushed into the application layer. Since MES delivery usually involves per-project customization, that slowed product iteration directly.

The solution

ETAO's head of engineering recognised quickly that converged workload support addressed exactly this: one database that satisfies the CRUD needs of a management information system, ingests high-frequency data quickly, supports complex queries, and speaks SQL would simplify the existing MES architecture enormously.

After the migration, a single MatrixOne cluster handles structured and semi-structured data together. The ETL jobs between the former systems were removed outright, several database technologies no longer need maintaining, and the access interface between database and application is unified — the MES returned to a minimal single-database architecture.

With data processing handed back to the database layer, developers could concentrate on the application, and features that had been queued too long started shipping.

Migration itself was not an obstacle. MatrixOne's high MySQL compatibility allowed the existing schema to be imported directly via SQL source, and the time-series tables in InfluxDB and document structures in MongoDB moved across as recommended — the whole exercise took under a week.

The outcome

The data architecture is dramatically simpler. ETAO Tech CTO Chen Ji: new developers no longer have to be trained on several databases — a working knowledge of MySQL is enough.

Analyses that previously required the application to gather data from multiple places and then build the logic are now written as SQL in the database, which improved both maintainability and performance, and MES project delivery moved considerably faster.

The team also began exploring work involving large models. MatrixOne carries relevant capability natively, which removes the need to introduce a separate vector database component for it.

The pattern is common across industrial manufacturing: facing mixed data workloads, engineers hand-assemble an architecture to support the business, and accumulate both code debt and operational debt in the process. That is precisely the situation a converged architecture is suited to.

Solution Architecture

Data sources
  • Device PLC and edge acquisition
  • MES people, material, process, equipment
  • High-frequency line time series
  • Traceability and quality history
MatrixOne
  • OLTP and OLAP on one engine
  • High-frequency time-series ingest
  • Native semi-structured JSON
  • Views replace cross-database ETL
  • MySQL-compatible migration path
Business applications
  • Report analysis and traceability
  • Predictive maintenance and process optimization
  • Minimal single-database architecture
  • Faster MES delivery

Related solutions and products