Configuring Ignition 8.3 QuestDB Historian Backup and Access

On Ignition 8.3.0 the Internal Historian - QuestDB (IHQ) runs embedded inside the gateway. It stores Tag History only. At initial release it has no redundancy, no store-and-forward on the local write path, and no integrated backup or restore. You build resiliency around it with file-level tools and gateway architecture. The sections below commission it in order, from scope through end-to-end proof. Each section ends with the check that must pass before you move to the next.

What will the QuestDB historian actually store on this gateway?

Operators see IHQ data in one place: tag history trends and tables driven by tags that have history enabled. The IHQ is a Tag History store. It does not replace a general-purpose SQL database. Transaction Groups, scripted inserts, and tables that Excel or reporting tools link to still need a real database connection.

Data path	Goes to IHQ?	Where it belongs
Tag History (tags with history enabled)	Yes	Internal Historian - QuestDB, local or on a dedicated historian gateway
Transaction Groups	No	SQL database connection
Scripted writes to custom tables	No	SQL database connection
Tables linked from Excel or BI tools	No	SQL database connection (IHQ is reachable read-only through a localhost PostgreSQL port, covered below)
Existing SQL historian partitions	No migration at release	Keep the existing SQL historian online and queryable

Strings are not a sizing concern. The IHQ places no special length limit on string values. QuestDB limits varchar fields to 268 MB and total column size to 218 TB. Disk capacity runs out long before either limit.

Check before proceeding: list every write path on the gateway. Every path that is not Tag History must have a SQL database connection configured and passing its connection test.

Should QuestDB share the gateway box or run on a dedicated historian gateway?

At release, Ignition supports only the embedded QuestDB historian. You cannot point Ignition at an external QuestDB server. External QuestDB support is planned but not available.

The long-standing guidance still applies: production databases belong on their own machine so the database cannot starve other processes of CPU and memory. With an embedded-only historian, you get that separation by moving the historian into its own Ignition gateway, not onto a bare database server.

Architecture	How it works	Use it when	Gaps
Single gateway, embedded IHQ	Gateway logs tag history into its own QuestDB	Small system; no SQL server or IT DBA available; you want a turnkey historian	Historian shares CPU/RAM with visualization and drivers; no S&F; no redundancy
Front-end gateway(s) + dedicated historian gateway	Front-end gateways use a Remote Historian over the Gateway Network; the historian gateway runs the IHQ	Medium systems; several gateways sharing one historian; you need the historian load isolated	Historian gateway is a single node until IHQ redundancy ships
Existing SQL historian (MSSQL, PostgreSQL, TimescaleDB, clustered SQL)	Gateway logs to a SQL server on separate hardware	The current system meets throughput, retention, and redundancy needs	Lower ingest rate than QuestDB; requires DBA management

When both the first two architectures would work, choose the dedicated historian gateway if more than one gateway produces history, or if the main gateway already runs heavy visualization and device loads. The dedicated historian node needs only the platform and the historian package. That licensing costs much less than a full gateway.

If a SQL historian on a separate cluster already handles the load, keep it. Examples include 3-day partitions across dozens of historian databases, or on-change storage for tens of thousands of tags. The history configuration on each tag carries through an 8.3 upgrade; it simply keeps pointing at the SQL provider instead of QuestDB. The IHQ gains you higher ingest and likely better compression. It does not provide a reason to rebuild a working system.

Check before proceeding: draw the chosen topology. For the dedicated option, confirm that each front-end gateway shows an active Gateway Network connection to the historian gateway.

How much RAM will the embedded QuestDB take from the gateway?

By default, the embedded historian uses 25% of available system memory. QuestDB memory-maps its column files and relies heavily on the OS page cache. This memory sits outside the gateway's Java heap. Size the box for heap plus QuestDB plus OS headroom, not heap alone.

Setting	Location	Effect
IHQ memory limit, percent form	Internal Historian - QuestDB settings on the gateway	Scales with host RAM; convenient on VMs that get resized
IHQ memory limit, absolute form	Same settings page	Fixed ceiling; predictable when the host also runs other services
Gateway JVM heap	Gateway memory configuration	Must fit alongside the IHQ limit without pushing the OS into swap

On a shared gateway, use the absolute form. This keeps a large trend query from growing the historian's footprint into the memory the drivers and sessions need. On a dedicated historian gateway, the percent form is fine because QuestDB is the main tenant.

Check before proceeding: run the heaviest expected trend query, such as a multi-week span across many pens. During the query, watch OS memory and swap and the gateway performance page. Swap activity or heap pressure during the query means you must lower the IHQ limit or add RAM.

How big will the disk get, and can ingest keep up with the tag count?

Two measured reference points from internal test datasets give a working bytes-per-point figure:

• 2.5 million points take approximately 176 MB uncompressed in the IHQ.
• The same data takes approximately 31 MB on ZFS with LZ4 compression.

Both are approximations.

Worked example. Assumptions: 88,259 tags, an average of one stored value every 5 s, every sample stored, and linear scaling from the test-dataset figures.

Quantity	Value
Points per day	88,259 x 17,280 = 1,525,115,520
Sustained ingest	~17,652 points/s
Disk per day, uncompressed	~107 GB
Disk per day, ZFS LZ4	~18.9 GB
Disk per year, uncompressed	~39 TB
Disk per year, ZFS LZ4	~6.9 TB

Ingest is not the limit here. Internal testing shows the IHQ handling 400K tags/second, a 7x write-throughput improvement over a SQL Historian on MSSQL. Retention is the limit. On-change storage with sensible deadbands cuts the points-per-day figure well below the fixed-rate case, so compute it from actual logged counts rather than tag count times rate.

Windows hosts cannot use ZFS. Size them with the uncompressed figure.

For comparison, one field TimescaleDB deployment logs about 2.5M points/day across just over 5,000 tags:

• Uncompressed, it uses about 300 MB/day (~107 GB/year).
• With native compression applied to chunks older than one week, it drops to about 15-17 MB/day (5.5-6.2 GB/year), roughly a 94% reduction.

TimescaleDB compression works on any OS. Partitioning and compression are configured in PostgreSQL, outside Ignition.

Check before proceeding:Divide one by the other to get your real bytes/point. Rerun the retention calculation with that number, then provision disk with margin.

How do remote gateways keep history when the historian link drops?

The Ignition 8.3 manual states that the IHQ does not support store-and-forward. On a single gateway writing locally, S&F is not needed. If the gateway cannot write to its own embedded historian, it almost certainly cannot write an S&F cache file to the same disk either.

S&F does matter in the split architecture. When the front-end gateway loses the Gateway Network link, or the historian gateway is down for maintenance, samples must buffer somewhere. That buffer lives in the Remote Historian on the sending gateway (previously called the Remote History Provider). The Remote Historian stores data on network disconnect or congestion and forwards it when the path recovers.

1. On the historian gateway, create the Internal Historian - QuestDB and confirm it reports running.
2. On each front-end gateway, create a Gateway Network outgoing connection to the historian gateway and approve it on the receiving side.
3. On each front-end gateway, create a Remote Historian that targets the historian gateway's IHQ.
4. Point the history provider on each historized tag to the Remote Historian, not to a local provider.
5. Size the Remote Historian's S&F capacity to cover the longest planned historian outage at your measured points/s.

Several front-end gateways can share one historian gateway. Each front end buffers independently.

Check before proceeding: break the Gateway Network link for a timed interval, for example by disabling the connection. Confirm the S&F buffer count on the front end rises. Restore the link and confirm the buffer drains to zero. Then open a trend covering the outage window and confirm it has no gap.

How do I back up the QuestDB data without corrupting it?

At 8.3.0, Ignition has no integrated backup or restore for the IHQ database. You can copy the database manually, but only with Ignition shut down. QuestDB writes to memory-mapped column files plus transaction metadata. A file copy taken while the gateway is writing can capture column files and metadata from different commit points, which gives you an unrestorable set.

The same applies to scheduled system-level backups. A file-based backup agent sweeping the live data directory produces exactly this torn copy. Either exclude the IHQ data directory from live file backups, or back it up with one of the methods below.

Method	Gateway downtime	Restores to	Notes
Cold file copy	Yes, for the copy duration	Same or replacement gateway	Simplest and consistent; schedule inside a maintenance window
ZFS snapshot (Linux)	No	Same dataset, or via send/receive to another host	Atomic point-in-time and crash-consistent; test restores
Native partition archiving	No	An external QuestDB instance, not back into the gateway at initial release	Gives read access to archived partitions outside Ignition
Live file-level backup agent	No	Unreliable	Exclude the IHQ directory

Cold backup procedure:

1. On the split architecture, confirm the front-end Remote Historians have S&F capacity for the window.
2. Stop the Ignition gateway service on the historian host.
3. Copy the entire IHQ data directory. Read its location from the historian configuration or the installation layout.
4. Start the gateway and confirm the IHQ reports running.
5. Confirm the front-end S&F buffers drain.

ZFS snapshot on Linux. The dataset names below are placeholders for your pool/dataset:

# one-time: put the IHQ data directory on its own dataset with LZ4
zfs set compression=lz4 tank/ignition-history

# scheduled: atomic snapshot, no gateway stop
zfs snapshot tank/ignition-history@hist-$(date +%Y%m%d-%H%M)

# off-host copy
zfs send tank/ignition-history@hist-20260928-0200 | ssh backuphost zfs receive backup/ignition-history

Placing the data directory on an LZ4 dataset gives you the ~12.4 B/point compression figure and cheap snapshots in one step.

Redundancy, integrated backup and restore, and moving partitions between storage locations are planned for early 8.3.x releases. Read the release notes of the exact 8.3.x build you install before choosing a method.

History collected on a beta build is not a supported migration source. Keep beta installs out of production. Carry beta data forward only through an in-place upgrade, if at all.

Check before proceeding: restore the latest backup onto a scratch gateway or VM. Start it, and trend a tag across a window you can also view on production. Both trends must match.

What happens to history when a redundant gateway pair fails over?

The embedded IHQ is not redundant at initial 8.3.0 release. Enabling gateway redundancy does not replicate the master's QuestDB to the backup. Support is planned, targeted within a few months of the initial release.

Operators see the result after a failover. Trends served from the backup do not contain the history the master wrote, and the master's store holds nothing for the period the backup was active. A master crash loses whatever the master's disk loses.

Requirement	Configuration that meets it today	Server count
Redundant visualization and devices, history survives failover	Redundant front-end pair with Remote Historian + S&F, feeding a dedicated IHQ historian gateway	3
Redundant visualization, no additional Ignition node	Redundant pair logging to a SQL historian on separate (ideally clustered) database hardware	2 + database
Single node, recoverable history	Embedded IHQ with ZFS snapshots or cold backups	1

The dedicated historian gateway is still a single node until IHQ redundancy ships. The S&F on each front end covers historian outages up to the buffer capacity, and snapshots or cold copies cover disk loss.

Check before proceeding: run a controlled failover. Trend the same tags across the failover window from both nodes and confirm continuous data from the historian gateway.

How do I query the QuestDB historian from outside Ignition, and why is a trend blank?

A blank or wrong trend comes from one of two different fault classes:

• Tag faults: the value was never stored.
• Binding faults: the value is stored but the query does not reach it.

Separate them before changing anything.

What the operator sees	Fault class	Where to look
Trend blank for one tag, fine for others	Tag	History enabled on the tag; history provider set to the IHQ or the Remote Historian; tag quality good
Trend blank for every tag from one front-end gateway	Tag/path	Remote Historian status; Gateway Network link; S&F buffer growing
Trend blank but a direct query shows rows	Binding	Chart pen path, provider name, and date range in the binding
Trend present but slow with MinMax aggregation	Expected behavior	MinMax on QuestDB runs two native-aggregation queries, versus one query plus Ignition-side aggregation on SQL historians
Gap only over a failover or outage window	Architecture	Redundancy and S&F sections above

For read expectations, a published internal benchmark gives a baseline:

• Query: 2 weeks from 600 million points spread evenly across 500 tags, one partition per day.
• Result shape: aggregates returning 300 points per query.
• Hardware: 11th-gen i7, 32 GB RAM, NVMe Gen3 storage.

QuestDB outran the SQL historians on all aggregates except MinMax, for the reason in the table.

To query the IHQ directly, set up a PostgreSQL-protocol database connection to it. This requires two extra parameters in the gateway's ignition.conf file. They are documented on the Gateway and Gateway Network Parameters page of the Ignition User Manual. Read the parameter names and the port from that page. The port is bound to localhost, so other machines cannot reach it. External tools such as Excel or reporting servers can reach it only through a proxy you place on the historian host. That proxy exposes raw history, so restrict it by firewall and credentials.

1. Add the two ignition.conf parameters from the manual page and restart the gateway.
2. From the historian host itself, connect with any PostgreSQL client to localhost on the configured port.
3. List the tables and run a row count bounded to one day.
4. If off-host access is required, add the proxy and repeat the test from the remote machine.

Check before proceeding: the one-day row count from the direct query must match the point count a Tag History table shows for the same tag and window.

How do I get history out of QuestDB into SQL, or keep existing SQL data visible?

A native export from the IHQ to a SQL database is not on the roadmap, because scripting already covers it. Build a scheduled gateway script with these steps:

1. Read tag history for a fixed, non-overlapping window.
2. Insert the rows into a SQL table through a named database connection.
3. Record the last exported timestamp, so a restart resumes where it stopped.

every N minutes:
  window_start = last_exported_ts
  window_end   = now - safety_lag    # let late/forwarded S&F data land first
  rows = read tag history (tag list, window_start..window_end, raw)
  insert rows into SQL table in one transaction
  on commit: last_exported_ts = window_end

Set the safety lag longer than your worst expected S&F drain time. Otherwise data forwarded late from a front end lands in a window that has already been exported.

Moving data between historian instances is a long-term roadmap item. No native migration or bridging tool exists for existing MSSQL or MySQL historians. Keep the old SQL historian provider configured and queryable alongside the IHQ, so trends spanning the cutover can still read pre-migration data.

Check before proceeding: for one tag and one day, compare three counts: the SQL row count from the export, the IHQ direct-query count, and the Tag History table count. All three must be equal.

How do I prove the whole historian chain end to end?

1. Pick three test tags on each front-end gateway: one analog on a fast rate, one boolean, and one string.
2. Confirm each tag's history provider points at the intended IHQ or Remote Historian.
3. Run the heaviest trend query during peak load. Confirm no swap on the host and no heap alarm on the gateway at the configured memory limit.
4. Break the Gateway Network link for a timed interval, restore it, and confirm the S&F buffer drains and trends show no gap.
5. Take a backup (cold copy or ZFS snapshot) and restore it to a scratch gateway. Confirm trends match production for the same window.
6. On a redundant front-end pair, fail over. Confirm continuous history from the historian gateway across the switch.
7. From the historian host, query the IHQ over the localhost PostgreSQL connection. Confirm a one-day count for each test tag matches the Tag History table count, and matches the SQL export count if you run one.

FAQ