Uncategorized

An enterprise platform powered by GenAI with automated extraction capabilities streamlines document handling for users. Utilizing Artificial Intelligence, the platform automatically identifies and extracts relevant data from various sources, reducing manual efforts and minimizing errors. This enables business users to perform easy and efficient extraction, saving time, improving data accuracy, and boosting overall productivity.

Users must have the Gen AI User policy to access the extraction capability. 

This guide will walk you through the steps on how to create an Extraction Agent.

1. Create an asset
2. Select a prompt template
3. Select a model and set model configurations
4. Provide the system instructions, parameters, output schema and examples
5. Run the model and view results
6. Publish the asset

Step 1: Create an asset

1. Head to the Gen AI Studio module and click Create Asset.
   
   
   
   
2. In the Create Gen AI asset window that appears, enter a unique Asset name, for example, “NACH_Mandate_Extractorr” to easily identify it within the platform.
   
   
   
   
3. Optional: Enter a brief description and upload an image to provide additional context or information about your Asset.
4. In Type, choose the Automation Agent and click Create.

Step 2: Select a prompt template

1. On the Gen AI Asset creation page that appears, choose Default Prompt template.
   
   
   
   

Step 3: Select a model and set model configurations

Select a Model

1. Select a model from the available List, considering model size, capability, and performance. Refer to the table to choose the appropriate model based on your requirements.  
   
   
   

LLM Model	Model Input – As per Platform configured	Model Output	Input Context Window(Tokens)	Output Generation Size(Tokens)	Capability and Suitable For
Azure OpenAI GPT 3.5 Turbo 4K	Supports Text	Text	4096	4096	Ideal for applications requiring efficient chat responses, code generation, and traditional text completion tasks.
Azure OpenAI GPT 3.5 Turbo 16K	Supports Text	Text	16384	4096	Ideal for applications requiring efficient chat responses, code generation, and traditional text completion tasks.
Azure OpenAI GPT – 4o	Supports Text	Text	128,000	16,384	GPT-4o demonstrates strong performance on text-based tasks like knowledge-based Q&A, text summarization, and language generation in over 50 languages. Also, useful in complex problem-solving scenarios, advanced reasoning, and generating detailed outputs. Recommended for ReAct
Azure OpenAI GPT – 4o mini	Supports Text	Text	128,000	16,384	A model similar to GPT-4o but with lower cost and slightly less accuracy compared to GPT-4o. Recommended for ReAct
Bedrock Claud3 Haiku 200k	Supports Text + Image	Text	200,000	4096	The Anthropic Claude 3 Haiku model is a fast and compact version of the Claude 3 family of large language models. Claude 3 Haiku demonstrates strong multimodal capabilities, adeptly processing diverse types of data including text in multiple languages and various visual formats. Its expanded language support and sophisticated vision analysis skills enhance its overall versatility and problem-solving abilities across a wide range of applications.
Bedrock Claude3 Sonnet 200k	Supports Text + Image	Text	200,000	4096	Comparatively more performant than Haiku, Claude 3 Sonnet combines robust language processing capabilities with advanced visual analysis features. Its strengths in multilingual understanding, reasoning, coding proficiency, and image interpretation make it a versatile tool for various applications across industries

Set Model Configuration

2. Click and then set the following tuning parameters to optimize the model’s performance.  For more information, see Advance Configuration.
   
   

Step 4: Provide the system instructions, parameters, output schema and examples

Provide System Instructions 

A system instruction refers to a command or directive provided to the model to modify its behavior or output in a specific way. For example, a system instruction might instruct the model to summarize a given text, answer a question in a specific format, or generate content with a particular tone or style.

1. Enter the system instructions by crafting a prompt that guides the agent in extracting the data. 
   
   
   
   

Add Parameters 

1. In the Parameter section, click Add.
   
   
   
   
2. Enter the following information.
   
   
   
   
   • Name: Enter the Name of the input parameter.
   • Type:  Choose File as the data type.
   • Description: Enter the Description for each of the input parameters. The description of the parameters ensures accurate interpretation and execution of tasks by the Gen AI Asset. Be as specific as possible.
     
3. Click against the input parameter to access settings and add input field settings.
4. Choose the required file formats (PDF, JPEG, JPG, TIFF, PNG) from the drop-down menu.
   
   
   
   
5. Select a chunk strategy for file inputs. The chunking strategy can be applied in Page, Words, and Block.
   
   
   
   
6. Click Save to proceed.

Define Output Schema

1. In the Output section, click Add to define the output schema for the Asset.
2. Enter the Variable Name, Type and Description for each of the output variables. Supported types include Text, number, Boolean, DateTime.
   
   
   
   

Provide Examples

Examples help the summarization task at hand to enhance the agent’s understanding and response accuracy. These examples help the agent learn and improve over time.

1. In the Examples section, click Add. 
   
   
   
   
2. Provide the Context and Answer in the example section.
   
   
   
   

Step 5: Run the model and view results

1. In the Debug and preview section, browse and add the required document.
   
   
   
   
2. Click Run to get the results for extraction in the required format. 
   
   
   
   
3. Review the generated output. Verify the classification by checking if the class is marked as true (indicating the data is classified as that class). If marked as false, the data is not classified as that class.
   
   
   
   
   
   • Click Reference  to view additional information or context about the classification results, such as the source data, detailed explanations, and relevant metadata.
     
     
     
     
     
   • Select the respective References to view its information.
     
     
     
     

Note: If you are not satisfied with the results then, try modifying the System Instructions and the description of the output variables. You can also try changing to a different model

View Trace 

1. If you wish to view the traces of the prompt and the result, click View trace.
   
   
   
   
2. In the Trance window that appears, review the trace.
   
   

Step 6: Publish the asset

1. Click Publish if the desired accuracy and performance for summarizing the content has been achieved.
   
   
   
    
2. In the Asset Details page that appears, write a description and upload an image for a visual representation.
   
   
   
   
3. Click Publish and the status of the Asset changes to Published then it can be accessed in the Gen AI Studio.
   
   
   

Note: Once the Asset is published, you can download the API and its documentation. The API can be consumed independently or used within a specific Use case. If you wish to consume this Asset via API, see Consume an Asset via API.

You can also consume this automation Asset in the Asset Monitor module. For more information, see Consume an Asset via Create Transaction.

The Purple Fabric Platform powered by Gen AI with automated classification capabilities streamlines the process of organizing and categorizing vast amounts of data for users. The Platform utilizes artificial intelligence to automatically classify data into predefined categories or groups based on its content, context, and attributes. By eliminating the need for manual sorting and classification, Purple Fabric platform enables business users to perform easy and efficient data classification. This enhances data management, improves data accuracy, and accelerates decision-making processes. Additionally, the platform’s intelligent algorithms can adapt to new data patterns and categories, ensuring flexibility and scalability to meet evolving business requirements.

Users must have the Gen AI User policy to access the classification capability. 

This guide will walk you through the steps on how to create a Classification Asset.

1. Create an asset
2. Select a prompt template
3. Select a model and set model configurations
4. Provide the system instruction, parameters, output schema and examples
5. Run the model and view results
6. Publish the asset

Step 1: Create an asset

1. Head to the Gen AI Studio module and click Create Asset.
   
   
   
   
2. In the Create Gen AI asset window that appears, enter a unique Asset name, for example, “Document_Classifier” to easily identify it within the platform.
   
   
   
   
3. Optional: Enter a brief description and upload an image to provide additional context or information about your Asset.
4. In Type, choose the Automation Agent and click Create.

Step 2: Select a prompt template

1. On the Gen AI Asset creation page that appears, choose Default Prompt template.
   
   
   
   

Step 3: Select a model and set model configurations

Select a Model

1. Select a model from the available List, considering model size, capability, and performance. Refer to the table to choose the appropriate model based on your requirements.  
   
   
   

LLM Model	Model Input – As per Platform configured	Model Output	Input Context Window(Tokens)	Output Generation Size(Tokens)	Capability and Suitable For
Azure OpenAI GPT 3.5 Turbo 4K	Supports Text	Text	4096	4096	Ideal for applications requiring efficient chat responses, code generation, and traditional text completion tasks.
Azure OpenAI GPT 3.5 Turbo 16K	Supports Text	Text	16384	4096	Ideal for applications requiring efficient chat responses, code generation, and traditional text completion tasks.
Azure OpenAI GPT – 4o	Supports Text	Text	128,000	16,384	GPT-4o demonstrates strong performance on text-based tasks like knowledge-based Q&A, text summarization, and language generation in over 50 languages. Also, useful in complex problem-solving scenarios, advanced reasoning, and generating detailed outputs. Recommended for ReAct
Azure OpenAI GPT – 4o mini	Supports Text	Text	128,000	16,384	A model similar to GPT-4o but with lower cost and slightly less accuracy compared to GPT-4o. Recommended for ReAct
Bedrock Claud3 Haiku 200k	Supports Text + Image	Text	200,000	4096	The Anthropic Claude 3 Haiku model is a fast and compact version of the Claude 3 family of large language models. Claude 3 Haiku demonstrates strong multimodal capabilities, adeptly processing diverse types of data including text in multiple languages and various visual formats. Its expanded language support and sophisticated vision analysis skills enhance its overall versatility and problem-solving abilities across a wide range of applications.
Bedrock Claude3 Sonnet 200k	Supports Text + Image	Text	200,000	4096	Comparatively more performant than Haiku, Claude 3 Sonnet combines robust language processing capabilities with advanced visual analysis features. Its strengths in multilingual understanding, reasoning, coding proficiency, and image interpretation make it a versatile tool for various applications across industries

Set Model Configuration

2. Click and then set the following tuning parameters to optimize the model’s performance.  For more information, see Advance Configuration.
   
   

Step 4: Provide the system instructions, parameters, output schema and examples

Provide System Instructions 

A system instruction refers to a command or directive provided to the model to modify its behavior or output in a specific way. For example, a system instruction might instruct the model to summarize a given text, answer a question in a specific format, or generate content with a particular tone or style.

1. Enter the system instructions by crafting a prompt that guides the agent in classifying the data.  
   
   
   
   

Add Parameters 

1. In the Parameter section, click Add.
   
   
   
   
2. Enter the following information.
   
   
   
   
   • Name: Enter the Name of the input parameter.
   • Type:  Choose File as the data type.
   • Description: Enter the Description for each of the input parameters. The description of the parameters ensures accurate interpretation and execution of tasks by the Gen AI Asset. Be as specific as possible.
3. Click against the parameter to access settings and add input field settings.
4. Choose the required file formats (PDF, JPEG, JPG) from the drop-down menu.
   
   
   
   
5. Select a chunking strategy for file inputs. The chunking strategy can be applied by Page, Words, or Block
   
   
   
   
6. Click Save to proceed.
   
   

Define Output Schema

1. In the Output section, click Add to define the output schema for the Asset.
   
   
   
   
2. Enter the following information.
   
   
   
   
3. In Variable Name, provide the names of the classes you wish to classify the data into.
4. In Type, select from any one of the following types.
   • Text 
   • Number
   • Boolean
   • DateTime
   • Signature 
     
5. In Description, enter the description for the parameter.

Note: The description of the parameters ensures accurate interpretation and execution of tasks by the GenAI asset.

Provide Examples

Examples help the classification task at hand to enhance the agent’s understanding and response accuracy. These examples help the agent learn and improve over time.

1. In the Examples section, click Add. 
   
   
   
   
2. Provide the Context and the Answer in the example section.
   
   
   
   

Step 5: Run the model and view results 

1. n the Debug and preview section, browse and add the required document.
   
   
   
    
2. Click Run to get the results for classification in the required format. 
   
   
   
   
3. Review the generated output. Verify the classification by checking if the class is marked as true (indicating the data is classified as that class). If marked as false, the data is not classified as that class.
   
   
   
   
4. Click Reference  to view additional information or context about the classification results, such as the source data, detailed explanations, and relevant metadata.
   
   
   
5. Select the respective References to view its information.

Note: If you are not satisfied with the results then, try modifying the System Instructions and the description of the output variables. You can also try changing to a different model.

View Trace 

• If you wish to view the traces of the prompt and the result, click View trace.
  
  
  
• In the Trance window that appears, review the trace.

Step 6: Publish the asset

1. Click Publish if the desired accuracy and performance for classifying the data has been achieved. 
   
   
   
    
2. Optional: In the Asset Details page that appears, write a description and upload an image for a visual representation.
   
   
   
   
3. Click Publish and the status of the Asset changes to Published then it can be accessed in the Gen AI Studio.
   
   
   

Note: Once the Asset is published, you can download the API and its documentation. The API can be consumed independently or used within a specific Use case. If you wish to consume this Asset via API, see Consume an Asset via API.

You can also consume this automation Asset in the Asset Monitor module. For more information, see Consume an Asset via Create Transaction.

Tools refer to the various components, including both custom-built solutions and third-party APIs, that can be customized and integrated to perform specific tasks or functions within a Gen AI Asset/Agent. These tools are essential for enhancing the capabilities of the Gen AI platform and enabling it to address diverse business needs effectively.

Users must have a Gen AI User policy to create and manage the tools. 

The Platform allows you to create the following types of tools:

• Custom tool
• API tool

Create custom tool

This is a component or module specifically developed by the enterprise to fulfill unique requirements or address specific challenges. By customizing tools, businesses can fine-tune their AI solutions to align closely with their objectives and tailor them to their operational needs.

These tools enhance the efficiency and effectiveness of AI-powered workflows, enabling enterprises to automate repetitive tasks, improve decision-making processes, and enhance customer experiences.

For example, a custom tool could be developed to automatically find the account holder with the highest saving amount. 

1. Head to Gen AI Studio module, choose Tools.
   
   
   
   
2. In the Tools section, click Create Tool.
   
   
   
   
3. In the Create Tool asset window that appears, enter a unique Name and Description.
   
   
   
   
4. In Type, choose Custom option and then click Create. 
5. On the Custom tool page that appears, write the Python code for the tool that you wish to create, and then click Test.
   
   
   
   
6. In the response section, you can review the response. .
   
   
   
   
7. If you are satisfied with the response, click Submit to create the custom tool. 

Note: You can test with static inputs and validate the response. Incase you want to pass dynamic input while integrating the custom tool as an agent, please define the inputs as dynamic variables. You may receive an error if you are using dynamic variables in the code. This doesn’t necessarily mean the code is not functioning properly, as the custom tool will be integrated with the LLM models and will fetch the inputs from them.

Create API tool

APIs play a crucial role in integrating third-party services or functionalities into the Gen AI platform. Leveraging APIs allows enterprises to access state-of-the-art AI capabilities without needing to build everything from scratch, accelerating development timelines and reducing costs.

For example, you could leverage an API that allows the user to retrieve and update information from internal databases or customer relationship management (CRM) systems.  

1. Head to Gen AI Studio module, choose Tools.
   
   
   
   
2. In the Tools section, click Create Tool.
   
   
   
   
3. In the Create Tool asset window that appears, enter a unique Name and Description.
   
   
   
   
4. Click Create.  
5. In the Tool page that appears, choose any one of the following methods and enter the URL. 
   
   • GET : Choose this method if you wish to obtain information from the database or from a process. 
   • POST: Choose this method if you wish to send information, either to add information to a database or pass the input of an Asset. 
   • PUT: Choose this method if you wish to manage/update the information in the database. 
   • DELETE: Choose this method if you wish to Delete the information from the database.

6. Click Test to check the response. 
7. You can also enter the Headers, Body and Parameters information from the respective tabs.

Add Headers

1. In the Headers tab, enter the Key, Value, and Description information.
   
   
   
   
2. You can also use the Constant/Variable field against the Key values.
   • Constant field : Activate this option to restrict the users to change the Key values while accessing the API. 
   • Variable field: Activate this option to allow the users to change the Key values while accessing the API.
3. Click    if you wish to add more fields. 
4. Select the respective check boxes against the Key, Value, Description information that you wish to process for this API call.   

Note: If you do not select the checkboxes, the keys will not be processed during the API call. 

Add Body 

1. In the Body tab,  use any one of the following options.
   
   • None: Choose this option if you wish to process the API without body information.
   • JSON: Choose this option if you wish to process the API with the JSON code. 
   • Form-data: Choose this option if you wish to process the API with the Key, Type, Value and Description information.
     
     
     
     
     • You can use the following options against the Type.
       
       • Text: Choose this option if you wish to update the Value information as text format. 
         
         • You can use the Constant/Variable field against the Type.
           • Activate Constant field to restrict the users to change the Key values while accessing the API. 
           • Activate Variable field to allow the users to change the Key values while accessing the API.
             
       • File: Choose this option if you wish to update the Value information as File
         
         
     • Click if you wish to add more fields. 
     • Select the respective check boxes against the Key, Type,  Value, Description information that you wish to process for this API call. 

Note: If you do not select the checkboxes, the keys will not be processed during the API call. 

Add Parameter

1. In the Parameter tab, enter the Key, Value information.
   
   
   
   
2. You can also use the Constant/Variable field against the Key values.
   • Activate Constant field to restrict the users to change the Key values while accessing the API. 
   • Activate Variable field to allow the users to change the Key values while accessing the API.
   • Click if you wish to add more fields. 
   • Finally, select the respective check boxes against the Key, Value, Description information. 
3. Select the checkbox against the keys that you wish to process for this API call. 

Note: If you do not select the checkboxes, the keys will not be processed during the API call.

8. Click Submit to create an API tool. 

Fine-tuning is the process of adjusting and optimising a trained Asset before it is published. This involves further training an Asset using an additional document set to boost its accuracy and confidence score.

To evaluate whether fine-tuning is necessary for an Asset, you can view the Accuracy Results page after Asset training is completed, which provides an overview of both correctly and incorrectly predicted entities information. By doing this, you can identify patterns or areas where fine-tuning can potentially improve the Asset’s performance. 

Note: Fine-tuning is only applicable for trained Assets before they are published, not to published Assets. If you wish to improve the performance of published Assets, you can proceed to retrain the Assets. For more information on retraining an Asset, see Retrain an Extractor Asset.

Step 1: Consider scenarios for fine-tuning 

The decision to fine-tune an Asset depends on your objectives, which are often fields and document specific.

1. Head to the Asset Studio page and select the trained Asset that you wish to fine-tune.
   
   
   
   
2. In the Accuracy Result page that appears, check the Asset’s overall accuracy rate, entity level accuracy and confidence score.
   
   
   
   

Things to know

Document type: In the context of Extractor Asset, a document type refers to the category or class that a document belongs to. For example, in an Extractor Asset, the document types can be “Invoice,” “Purchase Order,” “Receipt,” “Contract,” and more. Each of these represents a distinct category of documents. 

Entities: The entities refers to the fields and tables information.

Document Variation: Document variation refers to the different variations or instances within a specific document type. For example, various invoices could have different layouts, formats, or styles depending on factors like the vendor, company, or industry standards. 

Overall Accuracy: The overall accuracy represents the percentage of correct predictions made by the Asset across all Entities.

Entity level Accuracy: Entity level accuracy represents the percentage of correct predictions made by the Asset for individual entities in the test document set.

Confidence score: The confidence score is a measure of how confident the Asset is in its predictions for Entity information extracted from the documents.

3. You can consider the fine-tuning the Asset in the following scenarios:
   
   • To improve the overall accuracy of the Asset: Consider fine-tuning the Asset when the overall accuracy of the Asset is low. 
     
   • To improve the entity level Accuracy: Consider fine-tuning the Asset when the accuracy of certain entities is low.
     
   • To improve the accuracy for specific document variations: Consider fine-tuning the Asset for specific document variations with low accuracy. For example, if you’re creating an Extractor Asset to extract entities from invoices, and you notice low accuracy or confidence scores for invoices from specific vendors or invoice in certain formats, then you can initiate fine-tuning.
   • To improve the confidence score: Consider fine-tuning the Asset when the confidence score for certain entities or document variations is low. 
     

Step 2: Upload documents

After identifying areas for improvement in the Asset, it is recommended to have these required document sets for fine-tuning the Asset. If you have already uploaded the documents in Document Library, skip this step and proceed to Fine-tune.

Otherwise, upload the required documents in the Document Library. For more information about uploading documents, see Upload documents.

Step 3: Initiate fine-tune 

Note: It is important to be mindful that fine-tuning may also reduce the accuracy of the Asset when it is not properly performed with the appropriate document set and annotations.

1. On the Accuracy Result page, click Fine-tune.
   
   
   
   
2. In the Proceed to fine-tune window that appears, click Proceed.
   
   
   
   

Step 4: Select documents

1. In the Document Sets pane, select or search for the document set.
   
   
   
   
2. In the right page, select the required documents to fine-tune an Extractor Asset.
   
   
   
   

Note: Select a minimum of 10 documents to proceed for fine-tune. However, we recommend having a volume of 25 documents or more to provide a higher accuracy measure.

3. Click Proceed to annotate the documents.

Step 5: Annotate and train 

Data annotation is the process of labelling data to show the outcome you want your machine learning model to predict.

For more information on how to annotate fields, tables and sections, see Annotate field, Annotate a table and Annotate Section and Group.

Step 6 : Review results and validate

This step allows you to access the Asset’s predictions, accuracy, and confidence score. 

Additionally, you can utilise the Validate feature to evaluate the Extractor Asset’s performance on a new set of documents. 

For more information on reviewing the results and validation, see Review results and validate.

Step 7: Publish the asset

If the desired accuracy has been achieved, you can proceed to Publish the Asset. For more information on how to publish the Asset, see Publish the asset.

Note: Once the retrained Asset is published, you can download the API and its documentation. The API can be invoked independently or used within a specific Use case. If you wish to consume this Asset via API, see Consume an Asset via API.

It is recommended to use URL Aliases, if you wish to consume multiple versions of an Asset. It allows you to consume its different versions via a single API. For more information, see URL aliases.

You can also consume this asset in the Asset Monitor module. For more information, see Consume an Asset via Create Transaction.

The URL alias is a feature to create custom aliases for URLs associated with the APIs. This provides a more user-friendly experience when accessing the service URL.

The URL aliases allow the users to simplify and enhance the readability of the URLs used in the Asset APIs. For example, instead of using complex and lengthy URLs, users can create custom aliases that are more aligned with their Asset name. It converts the “{asset_version_id}” into user-friendly as “alias/{alias_name} ”.

Normal URL : {{baseUrl}}magicplatform/v1/invokeasset/{asset_version_id}/usecase

Alias URL : {{baseUrl}}magicplatform/v1/invokeasset/alias/{alias_name}

Additionally, URL aliases allow the users to consume different versions of an Asset via a single API. Users have the option to create the URL aliases, which helps them access multiple versions of an Asset without requiring to deploy their respective APIs. This reduces the need for multiple deployments to access different Asset versions.

This section in the Administration module allows you to create and manage the URL aliases on the platform. The following are the operations that you can perform in the URL aliases section and you must have the Administrator policy to perform these operations:

1. Create URL aliases 
2. Download API
3. Edit URL aliases
4. Delete URL aliases

Create URL aliases

1. Head to the Administration module and then select URL aliases.
   
   
   
   
2. In the URL aliases tab, click Create new.
   
   
   
   
3. On the URL new aliases page that appears, enter a unique Alias name.
   
   
   
   

Note: Modifying Alias name is not applicable once the URL alias is created. 

4. In the Asset Mapping section, select the desired Assets and its Version to consume.
   
5. Click Submit to create the URL alias.

Note: Once the URL alias is created, you can download the API and its documentation. For more information on how to download the API, see Download API. 

Download API

1. Head to the Administration module and then select URL aliases.
   
   
   
   
2. In the URL aliases tab, select the required Alias.
   
   
   
   
3. On the URL aliases information page that appears, click and select Download API to download its API. 
   
   
   
   

Note: You can consume the upgraded versions of an Asset using the same API. To consume different versions of an asset, you need to map the respective version in Asset Mapping.

The Downloaded API can be consumed independently. If you wish to consume this Asset via API, see Consume an Asset via API. 

Edit URL aliases

This section provides instructions on how to edit URL aliases on the platform. You can modify only the Asset Mapping information in this section. 

1. Head to the Administration module and select URL aliases.
   
   
   
   
2. In the URL aliases tab, select the Alias for which you wish to modify the information.
   
   
   
   
3. On the User alias information page that appears, click Edit.
   
   
   
   
4. Make the desired Asset mapping with the respective Assets and its Version then click Submit.
   
   
   
   

Note: Modifying Asset Mapping information will not affect the existing API information.

Delete URL aliases

This section provides instructions on how to delete existing URL aliases from the platform.

1. Head to the Administration module and select URL aliases.
   
   
   
   
2. In the URL aliases tab, select the URL alias you wish to delete.
   
   
   
   
3. On the URL aliases information page that appears, click and then select Delete.
   
   
   
   

Note: Deleting a URL alias is not possible while the Asset is being consumed.

Fine-tuning is the process of adjusting and optimising a trained Asset before it is published. This involves further training an Asset using an additional document set to boost its accuracy and confidence score.

To evaluate whether fine-tuning is necessary for an Asset, you can view the Accuracy Results page after Asset training is completed, which provides an overview of both correctly and incorrectly predicted document types. By doing this, you can identify patterns or areas where fine-tuning can potentially improve the Asset’s performance.

Note: Fine-tuning is only applicable for trained Assets before they are published, not to published Assets.

Step 1: Consider scenarios for fine-tune

The decision to fine-tune an Asset depends on your objectives, which are often specific Document types.

1. Head to the Asset Studio page and select the trained Asset that you wish to fine-tune.
   
   
   
   
2. In the Accuracy Results page that appears, check the Asset’s overall accuracy rate, Document type level accuracy and the confidence score.
   
   
   
   

Things to know

Document Type: In the context of Classifier Asset, a document type refers to the category or class that a document belongs to. For example, in a Classifier Asset, the document types could be “Invoice,” “Purchase Order,” “Receipt,” “Contract,” and more. Each of these represents a distinct category of documents. 

Document Variation: Document variation refers to the different variations or instances within a specific document type. For example, various invoices could have different layouts, formats, or styles depending on factors like the vendor, company, or industry standards. 

Overall Accuracy: The overall accuracy represents the percentage of correct predictions made by the Asset across all Document types.

Document type Accuracy: Document Type accuracy represents the percentage of correct predictions made by the Asset for individual Document types or categories in the test document set.

Confidence score: The confidence score is a measure of how confident the Asset is in its predictions for different Document types.

3. You can consider fine-tuning the Asset in the following scenarios:
   
   • To improve the overall accuracy of the Asset: Consider fine-tuning the Asset when the overall accuracy of the Asset is low.
     
   • To improve the Document type level accuracy : Consider fine-tuning the Asset when the accuracy for a certain Document type is low.
     
   • To improve the accuracy for specific document variations: Consider fine-tuning the Asset for specific document variations with low accuracy. For example, if you’re creating a Classifier Asset to classify invoices and purchase orders, and you notice low accuracy or confidence scores for invoices from specific vendors or purchase orders in certain formats, then you can initiate fine-tuning.
     
   • To improve the confidence score: Consider fine-tuning the Asset when the confidence score for certain Document types or document variations is low. 

Step 2: Upload documents

After identifying areas for improvement in the Asset, it is recommended to have these required document sets for fine-tuning the Asset. If you have already uploaded the documents in Document Library, skip this step and proceed to Fine-tune.

Otherwise, upload the required documents in the Document Library. For more information about uploading documents, see Upload Documents.

Step 3: Initiate fine-tune

Note: It is important to be mindful that fine-tuning may also reduce the accuracy of the Asset when it is not properly performed with the appropriate document set and annotations.

1. On the Accuracy Result page, click Fine-tune.
   
   
   
   
2. In the Proceed to fine-tune window that appears, click Proceed.
   
   
   
   

Step 4: Select documents

1. In the Document Sets pane, select or search for the document set.
   
   
   
   
2. In the right page, select the required documents to fine-tune a Classifier Asset.
   
   
   
   

Note: Select a minimum of 10 documents to proceed for fine-tune. However, we recommend having a volume of 25 documents or more to provide a higher accuracy measure.

3. Click Proceed to annotate the documents.

Step 5: Annotate and train

Annotation refers to the process of labelling documents against the Document types defined as part of the fine-tuning process.

For more information about annotating a Classifier Asset, see Annotate and Train.

Step 6: Review results and validate

This step allows you to assess the Asset’s predictions, accuracy, and confidence score. Additionally, you can utilise the Validate feature to evaluate the Classifier Asset’s performance on a new set of documents.

For more information on reviewing the results and validation, see Review results and validate.

Step 7: Publish the asset

If the desired accuracy has been achieved, you can proceed to Publish the Asset. For more information on publishing the Asset, see Publish the asset.