Datahub offloading inventory issue due to 'contents' exceeds the size limit of 32000 characters

Tech Community - Aug 17 '23 - - Dev Community

products versions - {Datahub version 10.11 and above}

Introduction

Datahub offloading inventory issue due to ‘contents’ exceeds the size limit of 32000 characters.

If the collection to be offloaded has JSON attributes consisting of more than 32,000 characters, its data cannot be offloaded. One specific case where this limitation applies is Cumulocity IoT’s application builder, which stores its assets in the inventory collection when being used.

Offloading jobs will throw the error as below.

image

As the datahub is using dremio to offload and query the data from datalake. This limitation is from dremio.

image

Steps to follow

To change the limitation at dremio end.

  1. Login to Dremio UI using administrator user

  2. Go to setting support and paste limits.single_field_size_bytes is support keys and click on show

    Image description

  3. After click on show it will display the default value as below:
    Image description

  4. This value can be change as per the requirement max till 256000
    Image description

  5. Once change click on the save.

Useful links | Relevant resources

Limits | Dremio Documentation

This page details limits for a variety of Dremio resources on a per-deployment basis.

favicon docs.dremio.com

Read full topic

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .