Inventory Overview

Inventory Introduction

Inventory is a feature that helps you manage objects in a bucket, and can be planned to replace the operations of the COS synchronous List API. Based on your inventory task configuration, COS can scan daily or weekly a specified object or objects with the same prefix in your bucket, export an inventory report, and save it as a CSV file to a bucket you specify. The file will list the stored objects and their metadata, and record the object attributes you need based on your configuration.
You can use the inventory feature for various purposes, including but not limited to the following basic ones:

• Reviewing and reporting the replication and encryption status of objects.
• Simplifying and accelerating business workflows and big data jobs.

  Note

  • You can configure multiple inventory tasks in one bucket. During the execution of these tasks, the object content will not be read directly, and only the attribute information such as object metadata will be scanned.

Inventory Parameters

After you configure an inventory task, COS will regularly scan the specified objects in your bucket based on your configuration, and export an inventory report, which is a CSV file. Currently, the following information can be recorded in an inventory report:

Inventory Configuration

Before configuring an inventory, you need to know two concepts:

• Source bucket: The bucket for which you want to enable the inventory feature.
  • Contains objects listed in the inventory.
  • Contains the configuration of the inventory.
• Destination bucket: The bucket where you want to store the inventory.
  • Contains an inventory list file.
  • Contains the Manifest files that describe the locations of the inventory list file.
    You need to follow the steps below to configure an inventory:

Specifying the Object Information to Be Analyzed in the Source Bucket

You need to tell COS the object information to be analyzed. Therefore, when configuring the inventory feature, you need to configure the following information in the source bucket:

• Select object versions: List all the versions of objects or list only the current version. If you select to list all the versions of an object, COS will record all historical versions of the object with the same name in the inventory report. If you select to list only the current version, COS will record only the latest version of the object.
• Configure the attributes of the objects to be analyzed: You need to tell COS what information from the object attributes needs to be recorded in the inventory report. Currently, supported object attributes include account ID, source bucket name, object file name, object version ID, whether it is of the latest version, whether it is a delete marker, object size, object's last modification date, ETag, object's storage class, cross-region replication tag, and whether it is a part in multipart upload.

Configuring the Storage Information for the Inventory Report

You need to tell COS at what frequency to export the inventory report, to which bucket you want the report to be stored, and whether the report needs to be encrypted. You need to configure the following information:

• Select an inventory export frequency: Daily or weekly. You can use this configuration to tell COS at what frequency it should execute the inventory feature.
• Select an inventory encryption mode: No encryption or SSE-COS. If you select SSE-COS encryption, we will encrypt the generated inventory report.
• Configure the inventory output location: You need to specify the bucket where to store the inventory report.

  Note

  The destination and source buckets should be in the same region, and can be the same bucket.

Usage Instructions

Configuring Inventory via the Console

To learn how to configure the inventory feature via the console, see Activate Inventory Feature in the console documentation.

Configuring Inventory via APIs

To use APIs to enable the inventory feature for a specified bucket, follow the steps below:

1. Log in to Tenant Side as the primary account, navigate to the Inventory Settings page of any bucket, and perform service authorization. Authorization is only required once for the same account.
2. Enable the inventory feature.

1. Enabling the Inventory Feature

Call the API to enable the inventory feature. For more information about the API, see PUT Bucket inventory. Here, the destination bucket to store the inventory file should be in the same region as the source bucket.

Inventory Report Storage Path

The inventory report and the relevant manifest files will be published to the destination bucket, in which the inventory report will be published in the following path:

destination-prefix/appid/source-bucket/config-ID/

The manifest files will be published in the following paths in the destination bucket:

destination-prefix/appid/source-bucket/config-ID/YYYYMMDD/manifest.json
destination-prefix/appid/source-bucket/config-ID/YYYYMMDD/manifest.checksum

The meanings of the paths are as follows:

• destination-prefix: The "destination prefix" you set when you configure the inventory. It can be used to group all inventory reports in a public location in the destination bucket.
• source-bucket: The name of the source bucket corresponding to the inventory report. This folder is added to avoid conflicts that may occur when multiple source buckets send their own inventory reports to the same destination bucket.
• config-ID: The "inventory name" you set when you configure the inventory. When multiple inventory reports are set in the same source bucket and sent to the same destination bucket, config-ID can be used to distinguish among different inventory reports.
• YYYYMMDD: Timestamp, including the start time and date of the bucket scan when the inventory report is generated.
• manifest.json: Means the manifest file.
• manifest.checksum: Means the MD5 of the content of the manifest.json file.
  Here, the manifest files include two files: manifest.json and manifest.checksum.

  Description

  The manifest files are described as follows:

  • Both manifest.json and manifest.checksum are manifest files. manifest.json describes the location of the inventory report, and manifest.checksum serves as the MD5 of the content of the manifest.json file. Each time a new inventory report is delivered, it comes with a new set of manifest files.
  • Each manifest contained in manifest.json provides the basic information of the inventory, such as the metadata. Such information includes the following items:
    • Name of the source bucket.
    • Name of the destination bucket.
    • Version of the inventory.
    • Timestamp, including the start time and date of the bucket scan when the inventory report is generated.
    • Format and architecture of the inventory file.
    • Object key, size, and md5Checksum of the inventory report in the destination bucket.

A sample of the manifest in the manifest.json file for a CSV-format inventory is shown below:

{
 "sourceAppid": "1250000000",
 "sourceBucket": "example-source-bucket",
 "destinationAppid": "1250000000",
 "destinationBucket": "example-inventory-destination-bucket",
 "fileFormat": "CSV",
 "listObjectCount": "13",
 "listStorageSize": "7212835",
 "filterObjectCount": "13",
 "filterStorageSize": "7212835",
 "fileSchema": "Appid, Bucket, Key, Size, LastModifiedDate, ETag, StorageClass, IsMultipartUploaded, ReplicationStatus",
 "files": [
  {
   "key": "cos_bucket_inventory/1250000000/examplebucket/inventory01/04d73d9debc73d9f0bf85af461abde6c.csv.gz",
   "size": "502",
   "md5Checksum": "7d40288a09c25b302ad6cb5fced54f35"
  }
 ]
}

Inventory Consistency

Inventory report by COS provides eventual consistency for PUT of both new objects and overwrites and for DELETE. Therefore, the latest added or deleted objects might not be included in the inventory report. For example, if you upload or delete an object when COS is executing an inventory task you configure, you might not see the results of your upload or deletion operations in the inventory report.
If you want to verify the object status before the execution, we recommend you call the HEAD Object API to extract the object metadata, or check the object attributes in the COS console.