Optimizing the performance of Lifecycle Query Engine (IBM LQE) reindexing

01 / 02Steps to take to improve Lifecycle Query Engine (LQE) performance

1. Reindex individual TRS data sources sequentially, addressing each one that requires reindexing separately. Avoid reindexing all of them simultaneously.
2. Once the reindexing of the initial TRS data source is complete, select “Pause Indexing.” This step is essential to halt the delta index operations, which typically occur every 60 seconds, thereby conserving server resources.
3. Evaluate the adjustment of the “Number of threads to fetch artifacts when adding or reindexing this data source” parameter while reindexing. The default setting is 2, but it can be augmented based on the number of virtual CPUs (vCPU). If your system boasts over 20 vCPUs and the data provider server can handle the additional load, consider elevating the thread count to 6-8. In the case of most multi-core CPUs, a temporary increase to 4 is generally deemed safe. Notably, on Intel systems, exceeding 4 threads tends to yield minimal enhancements in data retrieval speed.

Enlarge

4. Deactivate the LQE “Query Service” to prevent inadvertent queries from being directed to LQE by other Engineering Lifecycle Management (ELM) applications while the reindexing process is underway.

Clear ‘Query Service Enabled’ at lqe/web/admin/query.

Enlarge

5. Temporarily decrease LQE background processes. Ensure to revert these values afterward to avoid prolonged delays in updates.

Navigate to [Administration] > [Advanced Properties] > Click [Show Internal Properties] > [Access Control]. Select the Edit link next to [Internal Properties Configuration] for edits. While the properties are dynamic, a server restart is recommended.

Adjust the properties related to refresh to 2880 minutes to prevent them from coinciding with reindex operations.

Enlarge

Although adjusting these elements during a problem may not always be feasible, it is recommended to periodically assess the hardware and network configuration of the machine hosting the LQE index. These considerations often result in significant enhancements to reindexing performance.

Factors to review include:

• Solid State Drive (SSD) disks, which can reduce reindexing time by half due to their faster read/write speeds.
• The quantity and speed of dedicated virtual Central Processing Units (vCPUs).
• Network latency – plan for an optimal network setup to facilitate efficient transmission of HTTP messages.

For Windows platforms, contemplate implementing direct I/O mode during reindexing for potential performance gains.

02 / 02TRS validation or reindex?

IBM’s ELM 7.1 documentation describes validation as a lighter process that fixes differences between the TRS feed and the LQE index without rebuilding it. Reindexing deletes the indexed data and rebuilds it from scratch.

Run validation when

• Reports are missing artifacts, show stale or deleted artifacts, or show the same artifact more than once.
• An application view and a Jazz Reporting Service report show different counts.
• LQE or the Link Index Provider shows skipped resources.
• You want a routine health check. IBM recommends running validation weekly or monthly.

If you suspect the TRS feed itself is wrong, first run validation on the application side (application admin page > TRS Feed Diagnostics), then run validation on the LQE side.

Reindex when

• You are adding a new TRS feed.
• An indexing operation failed or was interrupted, for example by a server restart.
• Application-side and LQE-side validation cannot fix the inconsistency.
• The change log was truncated, a rollback was detected, or a rebase was done on the application.
• IBM Support tells you to. IBM recommends consulting IBM Support before reindexing.

Do not use validation to recover from a reindex that failed during the removal phase. Data may already be deleted, so start the reindex again. Once a validation has started, the only way to stop it is to shut down the application server, so run it when server usage is low.

Sources: Choosing between validation and reindexing (ELM 7.1), TRS Validation (jazz.net Deployment wiki).