Scenario

The Scenario tab allows selecting the data that you want to scrape - the Product Name, Product Description, SKU/Reference, Product Image, Price, Variations, etc.

Follow these steps to start the setup:

1. Enter the URL under the Product URL field.

If you selected to scrap from the URL, the link that you entered in the General tab will be automatically filled. If you want to check the other URL/product, you may enter the other URL there and click Refresh to see the product page. Note, that if you want to scrape the whole site, in this step you will have to enter the URL to a specific product to be able to add the fields that you want to scrape

• Inspect - click on the Inspect to open the HTML of your site. Select the section to see the needed XML in the code

In order to add a field (Title, Description, etc.), click to Inspect, and click on the product Name - the XML code for the Name is highlighted. On the right section, use the right-mouse click to Copy the XPath

Now click on the [+] button and select the Extract Field.

Under the Add button, you can see the following options: 

• Extract Field - click to add any field

• Extract Variation Field - to add the variations field

• Run the JavaScript - you can add a custom code for each field

• Wait - you can use it if you want to pause the scrapping

• Emulate Mouse Click (by Selector)  - is used to open for example the dropdown with variations or any other form that required an additional click on the page. In the Inspect section use the right-mouse click on the field that you want to scrape and select the Copy Selector

• Emulate Mouse Click (by XPath) -  is used to open for example the dropdown with variations or any other form that required an additional click on the page. In the Inspect section use the right-mouse click on the field that you want to scrape and select the Copy XPath. 

• Emulate Mouse Click ( as JS) - is used to open for example the dropdown with variations or any other form that required an additional click on the page. In the Inspect section use the right-mouse click on the field that you want to scrape and select the Copy JS Path. 

• Emulate [ENTER] or [DOWN] Key Action:
  [ENTER] - emulate the enter. For example, enter the drop-down to open it
  [DOWN] - allows moving down in the list. For example, move between the options inside the drop-down

• Scroll up - you can set the Pixels to scroll up

• Scroll  down -  set the Pixels to scroll down

• Scroll to the bottom - scroll to the bottom of the page

• If-then  - used for the specific Functions

• While (loop) - Function

• For (loop) - Function

• Input value (by XPath) - run the Javascript code

• Select (n) option item (by Xpath) -  set the Javascript and return the 1 ( True) or 0 ( false) on click

• Click as mouse on (n) Option-Item ( by XPath) - the mouse click emulation

In the Extract Field window, Select the export type, specify the Field Name (if needed) and insert the code that you copied instead of the  [=== ENTER XPATH HERE ===]

For example, how to add the Name:

• Select the field in the right section and copy the XPath

• Click on the [+] and select the Extract Field

• Select the Export type

• Insert the code

• Click Ok

You can add all the necessary fields in the same way.

Add / Delete / Preview scenario

The play scenario allows to see the result of the scrapping. Do not forget to click on Save.

If the result looks correct, you can proceed with a Full scrape and check the result. When the scrapping process is in progress, you can [Stop], [Pause], or [Edit scrape configuration] 

If you run the scrape a few times, you will get the confirmation notification

Click [No] to start from the begining or [Yes] to continue the previous scrape

On the Top menu, you can find the Navigation buttons

Once the process is finished, you can save the result to the file