Introduction

When working with large lists in Bubble, it's crucial to process data efficiently to avoid overloading your application and to respect capacity limits. Instead of using "Schedule API Workflow on a List," you can implement a recursive workflow in the Backend Workflows (also known as API Workflows). This approach allows you to process elements one by one, controlling the pace and avoiding load spikes.

Table of Contents

1. Overview of the Recursive Workflow

2. Steps to Implement the Recursive Workflow
   2.1. Create the Recursive Endpoint
   2.2. Configure the Endpoint Actions
   2.3. Initiate the Recursive Workflow

3. Advantages and Disadvantages
   3.1. Advantages
   3.2. Disadvantages

4. Conclusion

5. Workflow Implementation Summary

6. Final Considerations

1. Overview of the Recursive Workflow

A recursive workflow is a technique that allows you to sequentially process large lists of data by scheduling each workflow execution after the previous one has completed. This method is based on the workflow calling itself until all list elements have been processed, providing precise control over the execution pace.

2. Steps to Implement the Recursive Workflow

2.1. Create the Recursive Endpoint

Step 1: Access the Backend Workflows

• Navigate to the "Backend Workflows" tab in your Bubble editor.

• If you’re using Backend Workflows for the first time, enable them via Settings > API > Enable backend workflows.

Step 2: Create a New Endpoint

• Click on "Add an API endpoint".

• Name the endpoint (e.g., "process_large_list").

• Check the option "This workflow can be run without authentication" if needed (in production, consider adding security measures).

Step 3: Define the Endpoint Parameters
Add the following parameters:

• index (Number): Type: Number

• file_ids (List of texts): Type: List of texts

• file_names (List of texts): Type: List of texts

• file_mime_types (List of texts): Type: List of texts

• file_urls (List of texts): Type: List of texts

2.2. Configure the Endpoint Actions

Action 1: Create a New Record in the Database

• Add the "Create a new thing" action.

• Thing Type: GoogleDriveFile (or the name of your data type).

• Set the fields using the index:

  • fileId: file_ids:item# index

  • fileName: file_names:item# index

  • fileMimeType: file_mime_types:item# index

  • fileUrl: file_urls:item# index

Action 2: Schedule the Next Recursive Execution

• Add the "Schedule API Workflow" action (not "on a List").

• Configure the action as follows:

  • API Workflow: process_large_list

  • Scheduled date/time: Current date/time (you can add a delay if you wish to space out the executions)

  • Parameters:

    • index: index + 1

    • file_ids: pass the same list

    • file_names: pass the same list

    • file_mime_types: pass the same list

    • file_urls: pass the same list

  • Only when: index < file_ids:count
    (This condition ensures that the workflow continues recursively until all elements are processed.)

2.3. Initiate the Recursive Workflow

From your page workflow (e.g., when the onFilesSelected event is triggered):

1. Add a "Schedule API Workflow" action (not "on a List").

2. Configure the action:

   • API Workflow: process_large_list

   • Scheduled date/time: Current date/time

   • Parameters:

     • index: 1 (start at the first element)

     • file_ids: Element A's file_ids

     • file_names: Element A's file_names

     • file_mime_types: Element A's file_mime_types

     • file_urls: Element A's file_urls

3. Advantages and Disadvantages

3.1. Advantages

1. Precise Flow Control:

   • Allows you to add delays between executions to control the pace.

   • Processes each element individually, simplifying error handling.

2. Efficient Resource Usage:

   • Processing one element at a time prevents overloading your application.

   • Reduces the risk of hitting Bubble’s capacity limits, even with very large lists.

3. Scalability:

   • Suitable for lists of any size, with more predictable and stable performance.

4. Improved Error Handling:

   • Enables the implementation of specific logic to handle errors on individual elements without affecting the overall process.

3.2. Disadvantages

1. Increased Complexity:

   • Requires a more complex configuration and a good understanding of recursive workflows.

   • Can be more challenging to maintain and debug.

2. Longer Processing Time:

   • Processing elements sequentially may result in a longer total processing time compared to parallel processing.

   • Adding delays further increases the total time required.

3. Execution Limits:

   • Bubble imposes limits on the number of actions and workflows per minute, which could impact extremely large lists.

4. Tracking Progress:

   • It may be more challenging to implement progress indicators or provide user feedback on the processing status.

4. Conclusion

Using a recursive workflow is an effective solution for processing large lists in Bubble safely and efficiently. While it requires a more complex setup, the benefits in terms of control and scalability are significant for applications handling large volumes of data. It is advisable to supplement the implementation with visual diagrams or examples to enhance understanding.

5. Workflow Implementation Summary

1. Create the Recursive Endpoint:

   • Set up the necessary parameters in the Backend Workflows.

2. Configure the Endpoint Actions:

   • Create a database record using the index.

   • Schedule the next recursive execution by incrementing the index.

3. Initiate the Recursive Workflow:

   • From your page workflow, schedule the endpoint with the initial index.

4. Optimize and Adjust:

   • Consider adding delays and additional error handling as needed.

6. Final Considerations

• Security:
  Protect your endpoints in production by requiring authentication or other security measures.

• Monitoring:
  Use Bubble’s logging and monitoring tools to track performance and detect potential issues.

• Testing:
  Conduct extensive tests with different list sizes to ensure the workflow functions correctly in all scenarios.

• Visual Feedback:
  Consider implementing progress indicators or notifications to inform users of the processing status.

• Visual Documentation:
  Adding diagrams or visual examples can help clarify the process, especially for users new to the concept.