Config and Advanced Options

A monitor’s Config contains parameters in JSON format. The parameters are used to extract monitored content from a webpage.

How to view config?

You can navigate to the config of a monitor from the “Options” window. Here are the steps that you can follow to go to config:

1. Click on the “down caret” icon for a Monitor to view quick action items.

2. Click on the “Edit Options”. This will open the Options window.

3. Click the gear icon next to “open webpage selector” and the config view opens. This contains page options, selected and deselected elements of your selector.

You have the option to edit your monitor configuration either in the Config view or with a Config JSON.

Config parameters

When a monitor is added, Distill creates a config automatically. Following are the parameters in the config with a default value. You can change the default value to match your requirements.

1. index: Generated by default. Its value is based on selection. For selection inside an iframe, the index should match with that of iframe’s index.
2. excludes: List of selectors for excluded selections.
3. includes: List of selectors for included selections.
4. type: It can take any of three values depending on the selector type: “CSS”, “XPath” or “JS”. The default selector is CSS (XPath for browser extension).
5. expr: Selector expression for the selected or excluded text.
6. dynamic: It can be either set to “true” or “false”. True value loads dynamic content of webpages while false does not.
7. delay: Distill waits for the delay period before checking a page for a change. By default, it is set to 2. It can take values from 0 to 20. It is in seconds.
8. ignoreEmptyText: By default, it is set to true. In this case, when the selected element does not contain any text, it registers as error with error code “SELECTION_EMPTY” in the error-log. If you would like to consider empty text as a change, set it to false .
9. includeStyle: It is false by default. If it is set to true, it will include the webpage’s stylesheet in extracted data.

How to wait for a page to finish loading before extracting content?

Sometimes, web pages or specific sections of a page may take longer to load. If monitored using the default settings, Distill may not be able to find the content during its check, resulting in incomplete or erroneous data. To address this issue, you can increase the delay setting in the monitor’s config. This will instruct Distill to wait for the specified delay time (in seconds) while the page’s content loads before checking it. You can test out with different delay values and check what works best for that page. You can add a delay either in the configuration UI or add it in the Config JSON.

Method 1: To set a delay in Configuration:

1. When you press the gear icon next to the “Open webpage selector” button the Config UI opens up as below.

   

2. Expand the Page options and key in the delay interval in seconds in the Wait for Duration textbox

Method 2: To set a delay with Config JSON:

1. Open the monitor’s config JSON as shown below. The JSON editor opens up.

   

2. Search for the delay parameter. It has a default value of 2 or 0 depending on the device.

3. Change delay to some higher number (10 or 12) up to 20 and save the config.

   

How to make changes in the config in bulk?

You can use the Batch Edit feature to update the config of multiple monitors at once.

Here are the steps that you can follow:

1. Select the monitors for which you want to apply the new config.
2. Click on Batch Edit -> config.
3. Paste the new config and save.