Automation Basics

JavaScript SDK

Setting Up Your Environment

Before writing any code, you’ll need to set up your JavaScript environment and install the Herd SDK:

1. Make sure you have Node.js installed (version 14 or higher recommended)
2. Create a new project directory
3. Install the SDK using npm:

npm install @monitoro/herd

Initializing the Client

The first step in any automation is to initialize the Herd client with your API credentials:

// Import the Herd client
import { HerdClient } from '@monitoro/herd';

// Initialize the client with your API URL and token
const client = new HerdClient('your-token');

// Always initialize the client before using it
await client.initialize();

Note: Replace the token with your actual Herd API token from your dashboard.

Connecting to a Device

After initializing the client, you need to connect to a device (browser) that will perform the automation:

// Get a list of available devices
const devices = await client.listDevices();

// Connect to the first available device
const device = devices[0];

console.log(`Connected to device: ${device.id}`);

This code retrieves all devices registered to your account and connects to the first one. In a production environment, you might want to select a specific device based on its properties or availability.

Creating a Page and Navigating

Now that you’re connected to a device, you can create a new browser page and navigate to a website:

// Create a new page in the browser
const page = await device.newPage();

// Navigate to a website
await page.goto('https://example.com');

console.log('Successfully navigated to example.com');

The goto method loads the specified URL and waits for the page to load. By default, it waits until the page’s load event is fired, but you can customize this behavior with options.

Extracting Basic Information

One of the most common automation tasks is extracting information from web pages. Here’s how to extract basic elements:

// Extract content using CSS selectors
const content = await page.extract({
  title: 'h1',           // Extracts the main heading
  description: 'p',      // Extracts the first paragraph
  link: 'a'              // Extracts the first link text
});

// Display the extracted content
console.log('Extracted content:');
console.log(`Title: ${content.title}`);
console.log(`Description: ${content.description}`);
console.log(`Link: ${content.link}`);

The extract method uses CSS selectors to find elements on the page and extract their text content. This is a powerful way to scrape structured data from websites.

Proper Resource Management

Always remember to close resources when you’re done with them to prevent memory leaks:

// Close the page when done
await page.close();

// Close the client connection
await client.close();

Putting It All Together

Here’s a complete example that combines all the steps above into a single function:

import { HerdClient } from '@monitoro/herd';

async function runBasicAutomation() {
  const client = new HerdClient('your-token');
  
  try {
    // Initialize the client
    await client.initialize();
    console.log('Client initialized successfully');
    
    // Get the first available device
    const devices = await client.listDevices();
    if (devices.length === 0) {
      throw new Error('No devices available');
    }
    const device = devices[0];
    console.log(`Connected to device: ${device.id}`);
    
    // Create a new page
    const page = await device.newPage();
    console.log('New page created');
    
    // Navigate to a website
    console.log('Navigating to example.com...');
    await page.goto('https://example.com');
    console.log('Navigation complete');
    
    // Extract content
    console.log('Extracting content...');
    const content = await page.extract({
      title: 'h1',
      description: 'p',
      link: 'a'
    });
    
    // Display the extracted content
    console.log('\nExtracted content:');
    console.log(`Title: ${content.title}`);
    console.log(`Description: ${content.description}`);
    console.log(`Link: ${content.link}`);
    
  } catch (error) {
    console.error('Error during automation:', error);
  } finally {
    // Always close the client when done
    console.log('Closing client connection...');
    await client.close();
    console.log('Client connection closed');
  }
}

// Run the automation
runBasicAutomation();

Interacting with Web Pages

Now let’s explore how to interact with elements on a page. This includes clicking buttons, typing text, and handling forms.

Finding Elements

Before interacting with an element, you need to find it on the page:

// Find an element using a CSS selector
const searchBox = await page.$('input[name="q"]');

// Check if the element was found
if (searchBox) {
  console.log('Search box found');
} else {
  console.log('Search box not found');
}

The $ method returns the first element that matches the CSS selector, or null if no element is found.

Typing Text

To type text into an input field:

// Type text into an input field
await page.type('input[name="q"]', 'Monitoro Herd automation');
console.log('Text entered into search box');

The type method finds the element using the CSS selector and simulates typing the specified text.

Clicking Elements

To click a button or link:

// Click a button
await page.click('input[type="submit"]');
console.log('Search button clicked');

By default, the click method just clicks the element. If you want to wait for navigation to complete after clicking:

// Click and wait for navigation
await page.click('input[type="submit"]', { 
  waitForNavigation: 'networkidle2' 
});
console.log('Search button clicked and navigation completed');

The networkidle2 option waits until there are no more than 2 network connections for at least 500ms.

Waiting for Elements

Sometimes you need to wait for elements to appear on the page:

// Wait for an element to appear
await page.waitForSelector('#search');
console.log('Search results have loaded');

This is useful when dealing with dynamic content that loads after the initial page load.

Search Engine Example

Let’s put these concepts together in a search engine example:

async function searchExample() {
  const client = new HerdClient('your-token');
  
  try {
    await client.initialize();
    const devices = await client.listDevices();
    const device = devices[0];
    const page = await device.newPage();
    
    // Navigate to a search engine
    console.log('Navigating to Google...');
    await page.goto('https://www.google.com');
    
    // Type in the search box
    console.log('Entering search query...');
    await page.type('input[name="q"]', 'Monitoro Herd automation');
    
    // Submit the search form and wait for results
    console.log('Submitting search...');
    await page.click('input[type="submit"]', { 
      waitForNavigation: 'networkidle2' 
    });
    
    // Wait for results to load completely
    console.log('Waiting for search results...');
    await page.waitForSelector('#search');
    
    // Extract search result titles
    console.log('Extracting search results...');
    const searchResults = await page.extract({
      titles: {
        _$r: '#search .g h3',  // _$r extracts multiple elements
        text: ':root'           // For each match, get its text
      }
    });
    
    // Display the search result titles
    console.log('\nSearch Results:');
    searchResults.titles.forEach((result, index) => {
      console.log(`${index + 1}. ${result.text}`);
    });
    
  } catch (error) {
    console.error('Error:', error);
  } finally {
    await client.close();
  }
}