From Arkeia Documentation

Contents 1 Introduction 2 Technical Details 3 Starting A Block Optimization Process 3.1 Command Line Interface 4 Block Optimization Job Details 4.1 Web User Interface 4.2 Shell Console

Introduction

Block Optimization (UBP Recycling) provides two specific services.

Orphan Blocks

When retention periods for specific backup sets expire, the unique blocks to which the backup sets refer may become "orphans".
Here, an "orphan" is a deduplication block to which no backup set references. While the impact of orphans is generally negligible
at first, over long periods of time orphans can accumulate and adversely affect deduplication compression multipliers.
Block Optimization identifies, consolidates, and reclaims disk space used by orphaned blocks.

Duplicate Blocks

In case of "race conditions:, when two or more backup agents encounter a heretofore unknown block at almost exactly the same time,
more than one copy of the block may be sent by these clients to the backup server. The backup server will retain these
multiple copies until they are discovered during the block optimization process.

In summary, the overall benefit to using Block Optimization are:

• improved deduplication compression multipliers
• remove duplicate and redundant blocks
• decreases the size of the block pool and associate index files
• reclaim used disk space

Technical Details

Universal Block Pool (UBP) Recycling has no direct relation to Arkeia Backup Set recycling or backup job retention times.

The recycling process is implemented as a special Arkeia job who's input parameter is the UBP to recycle.
"arkbr" is a media server process that manages the UBP Recycling job which is interactively executed using the arkc (Command Line Interface) or by the "Start Block Optimization" option at the "Monitor Disk Storage" screen.

• Active Block Optimization Jobs can be viewed and aborted in the "Active Jobs" Screen.
• A running UBP recycling process can be interrupted by any job requiring access to the UBP. The UBP Recycling process will resume once the interrupting job releases the Pool.
• Cloud Replicated blocks are processed by the UBP Recycling process.
• The Block Optimization process treats v9.1 Source Side dedupe clients.

Starting A Block Optimization Process

Command Line Interface

The Block Optimization process can be starting manually using arkc (Command Line Interface) or automatically using cron.

Configure and Test arkc

Get the List of Disk Storage on the backup server

arkc -diskstorage -list

name=Target Disk 4
name=Target Disk 3
name=Target Disk 2
name=Target Disk 1

Start the Block Optimization process

/opt/arkeia/bin/arkc -ubp -br_start -D diskstoragename=[Target Disk 1]
jobid=4f43162c

NOTE: all Disk Storage that share a UBP with "Target Disk 1" will be processed.

More information about the "arkc -ubp -br_start" option can be obtained using the online akc documentation.

arkc -ubp -br_start -usage

Block Optimization Job Details

Details of the Block Optimization job can be viewed from the Server Journal File from the WebUI or via shell console.

Web User Interface

Reports > Server Journal

Shell Console

The /opt/arkeia/server/server.jl2 Journal files can be parsed using a shell console using the unique job ID (fourth column).

cat /opt/arkeia/server/server.jl2 | grep U1329794766

A 2012/02/20 19:26:07 I10100012 U1329794766 Start of blocks optimization for DiskStorage(s) 'Target Disk 1, Target Disk 3, Target Disk 4'
A 2012/02/20 19:26:08 I10000022 U1329794766 There is no UBP to purge!
A 2012/02/20 19:26:08 I10100020 U1329794766 Start of the optimization of UBP indexes located on clients machines
A 2012/02/20 19:26:08 I10100024 U1329794766 Optimizing the UBP indexes located on client 'cloudreplication.us.arkeia.com'...
A 2012/02/20 19:26:08 I02500192 U1329794766 The server blocks optimization generation ID associated to the UBP does not match the  client-side one. All existing client-side UBP indexes are being deleted...
A 2012/02/20 19:26:08 I10100025 U1329794766 The UBP indexes located on client 'cloudreplication.us.arkeia.com' have been optimized!
A 2012/02/20 19:26:09 I10100024 U1329794766 Optimizing the UBP indexes located on client 'HeatherNew'...
A 2012/02/20 19:26:09 E10100023 U1329794766 Cannot connect to 'HeatherNew'!
A 2012/02/20 19:26:09 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'HeatherNew'!
A 2012/02/20 19:26:09 I10100024 U1329794766 Optimizing the UBP indexes located on client 'win7.us.arkeia.com'...
A 2012/02/20 19:26:16 E10100023 U1329794766 Cannot connect to 'win7.us.arkeia.com'!
A 2012/02/20 19:26:16 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'win7.us.arkeia.com'!
A 2012/02/20 19:26:16 I10100024 U1329794766 Optimizing the UBP indexes located on client 'marketing.us.arkeia.com'...
A 2012/02/20 19:26:23 E10100023 U1329794766 Cannot connect to 'marketing.us.arkeia.com'!
A 2012/02/20 19:26:23 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'marketing.us.arkeia.com'!
A 2012/02/20 19:26:23 I10100024 U1329794766 Optimizing the UBP indexes located on client 'accounting.us.arkeia.com'...
A 2012/02/20 19:26:30 E10100023 U1329794766 Cannot connect to 'accounting.us.arkeia.com'!
A 2012/02/20 19:26:31 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'accounting.us.arkeia.com'!
A 2012/02/20 19:26:31 I10100024 U1329794766 Optimizing the UBP indexes located on client 'fedora-core16.us.arkeia.com'...
A 2012/02/20 19:26:38 E10100023 U1329794766 Cannot connect to 'fedora-core16.us.arkeia.com'!
A 2012/02/20 19:26:38 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'fedora-core16.us.arkeia.com'!
A 2012/02/20 19:26:38 I10100024 U1329794766 Optimizing the UBP indexes located on client 'fedora-core14.us.arkeia.com'...
A 2012/02/20 19:26:45 E10100023 U1329794766 Cannot connect to 'fedora-core14.us.arkeia.com'!
A 2012/02/20 19:26:45 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'fedora-core14.us.arkeia.com'!
A 2012/02/20 19:26:45 I10100024 U1329794766 Optimizing the UBP indexes located on client 'xenserver.us.arkeia.com'...
A 2012/02/20 19:27:06 E10100023 U1329794766 Cannot connect to 'xenserver.us.arkeia.com'!
A 2012/02/20 19:27:06 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'xenserver.us.arkeia.com'!
A 2012/02/20 19:27:07 I10100024 U1329794766 Optimizing the UBP indexes located on client 'sonylaptop'...
A 2012/02/20 19:27:28 E10100023 U1329794766 Cannot connect to 'sonylaptop'!
A 2012/02/20 19:27:28 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'sonylaptop'!
A 2012/02/20 19:27:28 I10100024 U1329794766 Optimizing the UBP indexes located on client 'ubuntu8.0.4-lotus.us.arkeia.com'...
A 2012/02/20 19:28:14 E10100023 U1329794766 Cannot connect to 'ubuntu8.0.4-lotus.us.arkeia.com'!
A 2012/02/20 19:28:14 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'ubuntu8.0.4-lotus.us.arkeia.com'!
A 2012/02/20 19:28:14 I10100024 U1329794766 Optimizing the UBP indexes located on client 'openfiler.us.arkeia.com'...
A 2012/02/20 19:28:21 E10100023 U1329794766 Cannot connect to 'openfiler.us.arkeia.com'!
A 2012/02/20 19:28:21 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'openfiler.us.arkeia.com'!
A 2012/02/20 19:28:21 I10100024 U1329794766 Optimizing the UBP indexes located on client 'lotus8.us.arkeia.com'...
A 2012/02/20 19:28:22 I02500192 U1329794766 The server blocks optimization generation ID associated to the UBP does not match the  client-side one. All existing client-side UBP indexes are being deleted...
A 2012/02/20 19:28:22 I10100025 U1329794766 The UBP indexes located on client 'lotus8.us.arkeia.com' have been optimized!
A 2012/02/20 19:28:22 I10100024 U1329794766 Optimizing the UBP indexes located on client 'backups.us.arkeia.com'...
A 2012/02/20 19:28:22 I10100025 U1329794766 The UBP indexes located on client 'backups.us.arkeia.com' have been optimized!
A 2012/02/20 19:28:22 I10100024 U1329794766 Optimizing the UBP indexes located on client 'mediaserver.us.arkeia.com'...
A 2012/02/20 19:28:22 I10100025 U1329794766 The UBP indexes located on client 'mediaserver.us.arkeia.com' have been optimized!
A 2012/02/20 19:28:22 I10100024 U1329794766 Optimizing the UBP indexes located on client 'oracle.us.arkeia.com'...
A 2012/02/20 19:28:23 I10100025 U1329794766 The UBP indexes located on client 'oracle.us.arkeia.com' have been optimized!
A 2012/02/20 19:28:23 I10100024 U1329794766 Optimizing the UBP indexes located on client 'mysql.us.arkeia.com'...
A 2012/02/20 19:28:23 I10100025 U1329794766 The UBP indexes located on client 'mysql.us.arkeia.com' have been optimized!
A 2012/02/20 19:28:23 I10100024 U1329794766 Optimizing the UBP indexes located on client 'vstorage-appliance.us.arkeia.com'...
A 2012/02/20 19:28:30 E10100023 U1329794766 Cannot connect to 'vstorage-appliance.us.arkeia.com'!
A 2012/02/20 19:28:30 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'vstorage-appliance.us.arkeia.com'!
A 2012/02/20 19:28:30 I10100024 U1329794766 Optimizing the UBP indexes located on client 'storage1.us.arkeia.com'...
A 2012/02/20 19:28:30 E10100031 U1329794766 Cannot execute the blocks optimization on 'storage1.us.arkeia.com'!
A 2012/02/20 19:28:31 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'storage1.us.arkeia.com'!
A 2012/02/20 19:28:31 I10100024 U1329794766 Optimizing the UBP indexes located on client 'storage.us.arkeia.com'...
A 2012/02/20 19:28:31 E10100031 U1329794766 Cannot execute the blocks optimization on 'storage.us.arkeia.com'!
A 2012/02/20 19:28:31 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'storage.us.arkeia.com'!
A 2012/02/20 19:28:31 I10100024 U1329794766 Optimizing the UBP indexes located on client 'oes2.us.arkeia.com'...
A 2012/02/20 19:28:38 E10100023 U1329794766 Cannot connect to 'oes2.us.arkeia.com'!
A 2012/02/20 19:28:38 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'oes2.us.arkeia.com'!
A 2012/02/20 19:28:38 I10100024 U1329794766 Optimizing the UBP indexes located on client 'sles11.us.arkeia.com'...
A 2012/02/20 19:28:39 E10100023 U1329794766 Cannot connect to 'sles11.us.arkeia.com'!
A 2012/02/20 19:28:39 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'sles11.us.arkeia.com'!
A 2012/02/20 19:28:39 I10100024 U1329794766 Optimizing the UBP indexes located on client 'hyper-v.us.arkeia.com'...
A 2012/02/20 19:28:39 E10100031 U1329794766 Cannot execute the blocks optimization on 'hyper-v.us.arkeia.com'!
A 2012/02/20 19:28:39 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'hyper-v.us.arkeia.com'!
A 2012/02/20 19:28:39 I10100024 U1329794766 Optimizing the UBP indexes located on client 'sql-server.us.arkeia.com'...
A 2012/02/20 19:28:47 E10100023 U1329794766 Cannot connect to 'sql-server.us.arkeia.com'!
A 2012/02/20 19:28:47 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'sql-server.us.arkeia.com'!
A 2012/02/20 19:28:47 I10100024 U1329794766 Optimizing the UBP indexes located on client 'vista.us.arkeia.com'...
A 2012/02/20 19:28:54 E10100023 U1329794766 Cannot connect to 'vista.us.arkeia.com'!
A 2012/02/20 19:28:54 E10100032 U1329794766 Cannot optimize the UBP indexes located on client 'vista.us.arkeia.com'!
A 2012/02/20 19:28:54 I10100021 U1329794766 End of the optimization of UBP indexes located on clients machines
A 2012/02/20 19:28:54 I10100013 U1329794766 End of blocks optimization