Pipeline Automation & Workflows

Automated Pipeline Runs #

Automatic Contact Enrollment in Workflows #

The pipeline automation system provides automatic contact enrollment in predefined workflows:

Automatic Pipeline Configuration:

• AutomaticPipeline Model: Defines which pipelines automatically apply to contacts
• Contact Type Association: Automatic pipelines are configured for Contact model type
• Pipeline Assignment: Multiple pipelines can be set to automatically enroll new contacts
• Team-Based Configuration: Automatic pipelines are configured at the team level

Contact Creation Triggers:

• Observer Integration: ContactObserver automatically triggers pipeline enrollment
• Immediate Enrollment: New contacts are enrolled in automatic pipelines upon creation
• Multiple Pipeline Support: Contacts can be enrolled in multiple automatic pipelines simultaneously
• Polymorphic Design: Pipeline system supports multiple model types beyond contacts

Enrollment Process:

• Automatic Detection: System detects automatic pipelines configured for contacts
• PipelineRun Creation: Automatic creation of pipeline runs for new contacts
• Status Initialization: Pipeline runs start with ‘pending’ status
• First Step Scheduling: Initial pipeline step is scheduled based on configured delay

Pipeline Run Management #

Pipeline runs provide comprehensive workflow execution management:

Pipeline Run Properties:

• Pipelineable Association: Polymorphic relationship to contacts or other models
• Pipeline Reference: Links to specific pipeline configuration
• Step Tracking: Tracks current and next step in pipeline execution
• Status Management: Comprehensive status tracking throughout workflow execution

Run Initialization:

• Automatic Scheduling: First step automatically scheduled with configured delay
• Status Setting: Initial status set to ‘pending’ for execution queue
• Step Number Tracking: Next step number initialized to start pipeline execution
• Timestamp Management: Next step execution time calculated and stored

Pipeline Steps and Delays #

Timed Actions and Workflow Progression #

The pipeline system supports sophisticated step-based workflows with timed execution:

Pipeline Step Structure:

• Step Number: Sequential numbering for ordered workflow execution
• Action Type: Defines what action to execute (send email, create task)
• Delay Configuration: Time delay before step execution (in seconds)
• Message Content: Content for email or other communication actions
• Additional Data: JSON storage for step-specific configuration

Step Action Types:

• Send Email: Automated email sending to contacts
• Create Task: Task creation for follow-up actions (planned functionality)
• Custom Actions: Extensible action system for additional workflow steps
• Action Parameters: Flexible data storage for action-specific requirements

Delay and Timing:

• Flexible Delays: Steps can have any delay duration specified in seconds
• Automatic Scheduling: Next step execution time calculated automatically
• Precise Timing: System processes pipelines every minute for accurate timing
• Time-Based Progression: Workflows progress based on configured time delays

Step Execution and Progression #

Pipeline step execution follows a structured progression model:

Step Execution Process:

• Status Updates: Pipeline run status updated to ‘running’ during execution
• Action Execution: Specific step actions executed based on step type
• Next Step Calculation: System calculates next step in sequence
• Completion Handling: Pipeline marked as ‘completed’ when all steps finished

Progression Logic:

• Sequential Execution: Steps execute in numerical order
• Automatic Advancement: System automatically advances to next step after execution
• Delay Application: Next step scheduled with configured delay
• Completion Detection: Pipeline completion detected when no more steps exist

Contact Observer Integration #

Automatic Pipeline Triggers on Contact Creation #

The contact observer system provides seamless automatic pipeline triggering:

Observer Functionality:

• Contact Creation Event: Observer automatically triggered when new contacts are created
• Automatic Pipeline Detection: System identifies automatic pipelines for contact model
• Pipeline Run Creation: Automatic creation of pipeline runs for new contacts
• Immediate Activation: Pipeline runs are created and scheduled immediately

Integration Benefits:

• Zero Manual Intervention: Pipelines start automatically without user action
• Consistent Application: All new contacts automatically enrolled in configured pipelines
• Scalable Automation: System handles any number of automatic pipeline configurations
• Reliable Execution: Observer pattern ensures consistent pipeline triggering

Observer Pattern Implementation:

• Event-Driven Architecture: Contact creation events trigger observer methods
• Automatic Pipeline Lookup: Observer queries automatic pipeline configurations
• Batch Pipeline Creation: Multiple pipeline runs created efficiently
• Error Isolation: Pipeline creation errors don’t affect contact creation

Manual Pipeline Assignment #

In addition to automatic enrollment, pipelines can be manually assigned:

Manual Assignment During Creation:

• Pipeline Selection: Users can select specific pipelines during contact creation
• Multiple Pipeline Support: Contacts can be assigned to multiple pipelines manually
• Pipeline Run Creation: Manual assignments create pipeline runs immediately
• Combined Automation: Manual and automatic pipelines work together

Pipeline Management During Editing:

• Pipeline Updates: Existing pipeline assignments can be modified during contact editing
• Pipeline Removal: Unwanted pipeline runs can be removed from contacts
• Pipeline Addition: New pipelines can be assigned to existing contacts
• Complex Pipeline Management: System handles complex pipeline assignment changes

Pipeline Processing #

Scheduled Execution of Pipeline Actions #

The pipeline processing system provides reliable scheduled execution:

Scheduled Processing:

• Every Minute Execution: Pipeline processing runs every minute for timely execution
• Pending Run Detection: System identifies pipeline runs ready for execution
• Batch Processing: Multiple pipeline runs processed efficiently in each execution cycle
• Error Handling: Individual pipeline run errors don’t affect other runs

Execution Queue Management:

• Time-Based Queuing: Pipeline runs queued based on next_step_at timestamps
• Status Filtering: Only ‘pending’ status runs are processed for execution
• Efficient Querying: Database queries optimized for fast pipeline run detection
• Load Management: System handles large numbers of pipeline runs efficiently

Processing Reliability:

• Error Logging: Comprehensive error logging for failed pipeline executions
• Exception Handling: Individual run failures don’t stop overall processing
• Status Tracking: Pipeline run status accurately tracked throughout execution
• Recovery Mechanisms: System designed for reliable pipeline execution

Pipeline Action Execution #

Pipeline actions are executed through a structured action system:

Action Processing:

• Action Type Detection: System identifies step action type for proper execution
• Email Actions: Send email actions processed through PipelineService
• Task Actions: Task creation actions prepared for future implementation
• Extensible Actions: Action system designed for easy extension

Email Action Processing:

• Email Sending: Automated email sending through pipeline service
• Email Content: Step message content used for email body
• Email Configuration: Additional data supports email-specific configuration
• Email Tracking: Email sending results tracked for pipeline analytics

Workflow Status Management #

Tracking Pipeline Run Status and Completion #

The workflow system provides comprehensive status management:

Pipeline Run Statuses:

• Pending: Pipeline run waiting for next step execution
• Running: Pipeline run currently executing a step
• Completed: Pipeline run finished all steps successfully
• Error Handling: Failed runs logged but don’t block other executions

Status Progression:

• Automatic Status Updates: Status automatically updated during pipeline execution
• Step-by-Step Tracking: Status changes tracked for each step execution
• Completion Detection: Automatic completion when no more steps exist
• Status Persistence: Pipeline run status stored for monitoring and analysis

Status-Based Processing:

• Pending Run Processing: Only pending runs are selected for execution
• Running Run Protection: Running runs are not reprocessed to prevent conflicts
• Completed Run Archival: Completed runs maintained for historical tracking
• Status-Based Filtering: Pipeline management filtered by run status

Workflow Monitoring and Analytics #

Pipeline workflow monitoring provides insights into automation effectiveness:

Execution Monitoring:

• Run Status Tracking: Monitor pipeline run status across all workflows
• Execution Timing: Track pipeline execution timing and delays
• Success Rate Monitoring: Analyze pipeline completion rates and success metrics
• Performance Analytics: Monitor pipeline processing performance and efficiency

Workflow Analytics:

• Pipeline Effectiveness: Analyze which pipelines are most effective
• Contact Engagement: Track contact response to automated workflows
• Workflow Optimization: Identify opportunities for workflow improvement
• Automation ROI: Measure return on investment for automated workflows

Error Analysis:

• Failure Tracking: Comprehensive tracking of pipeline execution failures
• Error Pattern Analysis: Identify common pipeline execution issues
• Recovery Monitoring: Track pipeline recovery and retry mechanisms
• System Health: Monitor overall pipeline system health and performance