CONCEPT: RockSolid Data Archive/Removal

The RockSolid application allows you to change how long you are keeping data collected from the SQL Server instances within the data repository.  Please see the settings topics on how to change these values:

RockSolid Data Removal Process

When you change the data storage parameters with RockSolid, on saving of the policy changes this may make a large amount of data within the RockSolid database expire.  For example, if at a group policy you reduce a retention policy from 720 days to 365 days, this may make hundreds of millions of individual rows of data now outside the policy window.  In order to ensure the removal of this data doesn't create a massive load on your database server, RockSolid will remove this data gradually.

The RockSolid data removal process uses a round-robin approach for each data feed repository, allocating a set amount of time per data feed and iterating over each data feed removing expired data in small batches (to avoid database contention).  This process repeats continuously while the RockSolid services are running as the data in RockSolid is retained as a sliding window (i.e. as new data enters, old data leaves).

Depending on the volume of data that is expired, the removal of this data may take several hours or in some cases days or weeks.  This is expected behavior as the risk of attempting to do this in a more aggressive manner may lead to resource contention, delaying the import of newly monitored data into the RockSolid repository.

RockSolid History Management

RockSolid has a history management process that can be scheduled on the RockSolid database server via the SQL Server agent.  To schedule the history management process, please create a new SQL Agent job which executes the following stored procedure in the RockSolid database:

exec [dbo].[rrsp_ANALYSISAGENT_HISTORY_RockSolidHistoryManagement]

This should be scheduled on a reoccurring basis, for example every 5 minutes.

History Management Configuration

The RockSolid history management process will override any configuration settings in the RockSolid application, clearing tables based on a hard set retention "number of days".  This process is controlled via a management view, the vwRockSolidHistoryManagement view.   To create this view execute the following TSQL on the RockSolid database:

create view vwRockSolidHistoryManagement
as
select RockSolidHistoryManagementKey, TableName, h.GobalHistoryRetentionDays as HistoryRetentionDays from RockSolidHistoryManagement h
go

To change the history retained in a particular table, please update the HistoryRetentionDays column, passing the key.  For example, to change the history retention for the EventHistory table to 365 days run the following TSQL:


update vwRockSolidHistoryManagement
set HistoryRetentionDays=365
where RockSolidHistoryManagementKey='EVENTS_CLOSED'

Using this process you can customize the amount of data reteained in key history tables, however this is commonly set to a consistent retention to avoid complexity. 

NOTE: Please use this with caution.  RockSolid will remove data older than the retention period so please ensure you set this correctly in line with your business requirements.

NOTE: If you configure a shorter history retention in the RockSolid application data will be remove to that retention policy rather than the one set here.  i.e. the smallest retention period will always be applied.

 

 

 

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.