1. Module Overview

• The SharePoint module within the Knowledge Hub enables users to link their SharePoint drives that serve as knowledge sources for creating embeddings for Generative AI-powered bots.
• This module allows users to link the data within their SharePoint drives by periodically syncing the drive data with our s3 bucket.
• Bots can leverage the documents to provide accurate and contextually relevant responses during interactions.

2. Functional Specifications and Terminology

• Required Fields:
  • SharePoint URL:
    • The URL to the SharePoint drive:
      Example:
      if the URL looks like this: https://gmsolutionsindia.sharepoint.com/sites/Talkk_SharepointTest/Documents_test/Forms/AllItems.aspx
      
      the URL that should be entered into the field should be: https://gmsolutionsindia.sharepoint.com/sites/Talkk_SharepointTest/Documents_test/
      
      The module will extract the hostname (gmsolutionsindia.sharepoint.com), site path (/sites/Talkk_SharepointTest/) and the drive (Documents_test)
  • Client ID:
    • The Client ID is a unique identifier for your application, used to authenticate it to your SharePoint environment, often alongside a Client Secret.
      
      Generating a SharePoint Client Secret
      • Navigate to the registration page: Go to your SharePoint site and navigate to the _layouts/15/appregnew.aspx page.
      • Create a new client ID: Click Create a new client ID and client secret.
      • Enter application details:
        • Title: Provide a descriptive title for the application.
        • App Domain: Input a domain for your app, such as www.localhost.com.
        • Redirect URI: Enter the URI where the authentication code will be sent, for example, https://www.localhost.com.
      • Click Create: This will generate a new Client ID and Client Secret.
      • Copy and store securely: Immediately copy the Client ID and Client Secret and store them in a secure location, as the secret cannot be retrieved later. 
    • Client Secret:
      • A SharePoint client secret is a credential, similar to a password, used by applications to authenticate and access SharePoint Online resources.
    • Tennent ID:
      • A SharePoint Tenant ID is a unique, 32-character global identifier for your organization’s Microsoft 365 subscription that links all your services.
        
        How to find it:
        • Go to the Azure admin center or your Microsoft 365 admin center.
        • Navigate to Identity and then Overview.
        • Look for the Tenant ID field, which contains your unique 32-character alphanumeric identifier.

3. Operational Details

The user has to enter the SharePoint URL, Client ID, Client Secret Key, Tennent ID on the page.

The cron job will trigger at 2300hrs everyday and all sync all the SharePoint drives for all the companies and groups.

The flow of the job is as follows:

1. The scheduler triggers the cron to fire
2. The cron fires and traverses through all the companies and groups and starts the sync service for each groupID async
3. The service syncs the SharePoint drive with the s3 bucket
   • The service will run async for each groupId
   • Populate the config from the database
   • Load state and delta link if exists else create a new state and delta file for the respective groupID
   • Compare the delta against the existing state and perform operations (add, update or delete) based on the incoming delta.