From Manual to Automated Testing: My Journey Creating a Bot for Website Testing

13 min readMar 9, 2023

Introduction

Website testing is a critical aspect of software development that ensures a website functions as expected and meets the needs of its users. However, manually testing a website can be time-consuming and error-prone, particularly when dealing with complex websites or frequent updates. This is where automation comes in — automating the website testing process can help to improve the speed, accuracy, and efficiency of testing while freeing up time and resources for other important tasks.

Automated website testing involves using software tools to run predefined test cases and scripts, comparing the actual results to the expected results, and generating reports. This allows developers to identify and fix issues quickly before they impact users. Furthermore, automated testing can help to improve the scalability of testing, allowing for testing to be performed more frequently and across a wider range of devices and configurations.

In this article, I will share my personal experience creating a bot for website testing. By automating the testing process, I was able to significantly reduce the time and effort required for testing, while also improving the accuracy and scalability of the testing process. In this journey, others will be inspired to consider automating their own website testing processes, ultimately leading to better software quality and user experiences.

Solving a Problem: How I Fixed

Manual testing is a critical aspect of software testing, and I have had extensive experience with it during my career as a software tester. In my experience, manual testing involves the process of checking software applications for defects or errors by going through each step in a predefined test case manually.

One of the significant challenges I faced during manual testing was the time-consuming nature of the process. It could take hours, days, or even weeks to manually test a single application, depending on its complexity. Additionally, manual testing requires human testers to repeat the same steps over and over again, which can lead to boredom and potential errors due to fatigue.

Another challenge of manual testing is the high likelihood of human error. Despite the best efforts of testers, there is always a risk of missing critical defects, either due to oversight or inadequate test coverage. Additionally, manual testing is prone to subjectivity, meaning that different testers may have varying opinions on whether a particular feature or functionality is working correctly.

Let’s Get to the code

To build my testing bot, I used a combination of open-source tools and technologies. Here are the main steps I followed:

Define the scope of your testing bot: Before you begin building your bot, you need to determine what tasks it will be performing. This will help you determine what tools and technologies you need to use, and what kind of framework you should be building.
Choose your programming language: I decided to use Python as my programming language because it has a lot of great libraries and tools for web scraping and automation.
Choose your web automation tool: I used Selenium WebDriver to interact with websites and automate my testing tasks. Selenium is a popular open-source tool for web automation and testing. another good thing Selenium has is it can work headless and you can put them into Docker containers
Write your code: I started by writing the code to initialize the Selenium WebDriver and navigate to the website I wanted to test. From there, I used the Selenium API to interact with the website, clicking buttons, filling out forms, and reading data.
Implement test cases: Once I had the basic code written, I started implementing specific test cases. This involved writing code to simulate different scenarios, such as filling out a form correctly, filling out a form with errors, or clicking on different buttons on the website.
Debugging and troubleshooting: Along the way, I faced a number of challenges, such as website changes or unexpected errors. To overcome these, I used the debugging and troubleshooting tools that are built into Python and Selenium, as well as other open-source libraries.
Test and refine: Once I had my bot up and running, I began using it to test the website. I made adjustments to the code as needed to improve its accuracy and efficiency.

One of the main challenges I faced was ensuring that the bot was reliable and accurate. This involved a lot of trial and error, as well as debugging and troubleshooting. I also had to ensure that the bot was scalable, meaning that it could be used to test multiple websites and perform a wide range of tasks as I write a decorator like that so I can handle errors and network problems happen:

"""
This is a snippt code which tries to run a function n times and if it ouccurd
an error, it will log every step, take an screen shot and etc ...
"""
MAX_TRY_CNT = 10 # Numbers to try a work
class Surfer:
    @staticmethod
    def try_it(func):
        def inner(self):
            t0 = datetime.now()
            func_name = func.__name__.replace("_", ' ')
            try_cnt = 0
            while try_cnt <= MAX_TRY_CNT:
                try:
                    logger.info(f"Trying to {func_name} - {try_cnt} [Info]")
                    output = func(self)
                except Exception as e:
                    #self.take_screenshot(func_name)
                    logger.warning(f"{e} Happened")
                    pass
                else:
                    t1 = datetime.now()
                    logger.info(f"Succeed {func_name} [Success]")
                    return output, (t1 - t0).seconds
                time.sleep(1)
                try_cnt += 1
            else:
                logger.error(f"Max try captured while {func_name} {try_cnt} >= {MAX_TRY_CNT} [Error]")
                self.take_screenshot(func_name)
                self.driver.delete_all_cookies()
                self.driver.close()
                raise SurferException()

        return inner

This code defines a class Surfer with a static method try_it and an instance method take_screenshot. The try_it the method takes a function as an argument and returns a new function that wraps the original function with error handling and retries logic. If the wrapped function succeeds, the output and time taken are returned. If it fails after the maximum number of retries, a screenshot is taken, the browser is closed, and a SurferException is raised.

The take_screenshot the method takes a function name as an argument and captures a screenshot using the save_screenshot method of the driver attribute of the Surfer instance.

Note that the code references a MAX_TRY_CNT the variable that is not defined in the code snippet provided, so it is assumed to be defined elsewhere.

now let's see some of the functions and actions in Snapp! Express simple flow:

@try_it
def _fill_basket(self):
    """
    Fill basket
    :return:
    # TODO: Handle if the category doesn't have enough product
    """
    # IF vendor is closed, choose another time delivery
    try:
        self.driver.find_element(By.XPATH, '//*[@id="modal-root"]/div[13]/div/div[3]/button').click()
    except Exception:
        pass

    self._scroll()
    buttons = self.driver.find_elements(By.TAG_NAME, 'button')
    for button in buttons:
        self._clear()
        if button.text == "افزودن":
            button.click()

    self.sleep(0.5)

    try:
        check_out = self.driver.find_element(By.XPATH, '//*[@id="pr-2"]/div/div[3]/button')
    except Exception:
        raise Exception("Basket is not fulled yet...")

@try_it
def _go_to_checkout(self):
    if self.driver.current_url == "<URl of Checkout>":
        return  True

    check_out = self.driver.find_element(By.XPATH, '//*[@id="pr-2"]/div/div[3]/button')
    check_out.click()

@try_it
def _go_to_payment(self):

    # Continue
    continue_button = self.driver.find_element(By.XPATH, '//*[@id="pr-2"]/div/div/button')
    continue_button.click()

    if self.driver.current_url != '<ULR of payment page>':
        raise SurferException("Didn't got payment")

@try_it
def _go_to_payment_gateway(self):
    # Go to payment
    self.sleep(2)
    old = self.driver.current_url
    payments = []
    try:
        payments.append(self.driver.find_element(By.XPATH, '//*[@id="payment-bank-AP_Web"]'))
    except Exception:
        pass
    try:
        payments.append(self.driver.find_element(By.XPATH, '//*[@id="payment-bank-saman"]'))
    except Exception:
        pass
    try:
        payments.append(self.driver.find_element(By.XPATH, '//*[@id="payment-bank-parsian"]'))
    except Exception:
        pass

    self._clear()
    random_gateway = random.choice(payments)
    gateway_name = random_gateway.text.split('\n')[1]

    logger.info(f"Choosing a payment randomly ->  {gateway_name}")

    random_gateway.click()

    self.sleep(2)

    payment = self.driver.find_element(By.XPATH, '//*[@id="payment-submitOrder"]')
    payment.click()

    self.sleep(3)

    if self.driver.current_url == old:
        raise Exception("Didn't got payment gateway")

@try_it
def _cancel_order_payment(self):
    # Cancel payment
    cancel_pay = None
    old = self.driver.current_url
    try:
        self.driver.find_element(By.XPATH, '//*[@id="cancel-btn"]').click()
    except Exception:
        pass
    try:
        self.driver.find_element(By.XPATH, '//*[@id="button-payment-cancel"]').click()
    except Exception:
        pass
    try:
        self.driver.find_element(By.XPATH, '//*[@id="btnCancel"]').click()
    except Exception:
        pass

    self.sleep(3)
    if self.driver.current_url == old:
        raise Exception("Didn't canceled from payment gateway")

    #self.take_screenshot("Success")

@try_it
def _choose_address(self):
    """
    Choose address if needed
    :return: None
    """
    self.driver.find_element(By.XPATH, EL.ADDRESS_BTN).click()

This code is a Python class named Surfer, which seems to be an implementation of a web scraper using Selenium. It contains a static method called try_it that takes another function as an argument and returns a new function that wraps the original one. This new function has a try-except block that tries to call the original function and returns its result if successful. If the original function raises an exception, the new function logs a warning message and retries the original function until a maximum number of attempts is reached, or the function succeeds.

The class has an instance method named ensure_driver that returns a webdriver.Chrome instance. This method uses a static variable called DRIVER to store a reference to the driver instance if it has been created previously; otherwise, it creates a new instance using the webdriver.Chrome constructor and the chrome_options object.

Other methods of the class include _get_url, _set_login_cookie, _choose_vendor, _scroll, _clear, _choose_random_category, among others, which perform different tasks using Selenium to interact with a web page. All of these methods use the try_it decorator to handle exceptions and retry the function if necessary.

Finally, the class has a sleep static method that sleeps for a random time between 3 and 5 seconds and a take_screenshot method that takes a screenshot of the current web page and saves it to a file.

You can also create your actions using XPATH or CSS class selectors,

XPath is a language for selecting nodes in an XML document, and is commonly used to locate elements in HTML documents. It uses a path-like syntax to traverse the HTML tree and identify elements based on their attributes and position relative to other elements.

Here’s an example of using an XPath expression to locate a specific button element on a web page:

from selenium import webdriver

driver = webdriver.Chrome()
driver.get("https://www.example.com")

# Locate the button element using an XPath expression
button = driver.find_element_by_xpath("//button[@id='submit-button']")

# Click the button
button.click()

# Close the browser window
driver.quit()

In this example, the XPath expression "//button[@id='submit-button']" selects the button element with an id attribute of submit-button. The find_element_by_xpath the method returns the first element that matches the XPath expression.

Get results, using Grafana, Prometheus, and Dockerize the project

Dockerizing project:

as we have _ensure_driver method and we added the headless and no graphic options to our driver:

@try_it
def ensure_driver(slef):
    """
    Ensure driver
    :return:
    """
    from selenium.webdriver.chrome.options import Options
    options = Options()
    options.add_argument('--headless')
    options.add_argument("--no-sandbox")
    options.add_argument("--window-size=1920,1080")
    options.add_argument("--disable-setuid-sandbox")
    options.add_argument("--remote-debugging-port=9222")
    options.add_argument("--disable-dev-shm-using")
    options.add_argument("--disable-extensions")
    options.add_argument("--disable-gpu")
    options.add_argument("start-maximized")
    options.add_argument("disable-infobars")
    if not DRIVER:
        try:
            return webdriver.Chrome(options=options, executable_path="./chromedriver")
        except Exception:
            return webdriver.Chrome(options=options, service=Service(ChromeDriverManager().install()))
    return DRIVER

The function then checks whether the global variable DRIVER is already defined. If it is, the existing driver instance is returned. Otherwise, the function attempts to create a new driver instance using the webdriver.Chrome class. If this fails, it attempts to create a new driver instance using the ChromeDriverManager class from the webdriver_manager module, which will automatically download and install the appropriate version of the ChromeDriver executable.

Finally, the function returns the newly created driver instance or the existing one if it already exists.

it can help us run the code in a docker container:

FROM python:latest

WORKDIR /code


RUN curl https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb --output google-chrome-stable_current_amd64.deb
RUN curl https://chromedriver.storage.googleapis.com/109.0.5414.74/chromedriver_linux64.zip --output chromedriver_linux64.zip

COPY . .

RUN unzip chromedriver_linux64.zip

RUN apt-get update && apt-get install -y ./google-chrome-stable_current_amd64.deb


RUN pip install -U pip
RUN pip install -r requirements.txt

CMD ["python", "main.py"]

This Dockerfile downloads the latest version of Python, sets the working directory to /code, and then proceeds to download the stable version of Google Chrome and the corresponding version of the ChromeDriver for Linux.

Next, it copies the current directory ( codes, driver ) into the working directory.

The unzip command is then used to extract the Chromedriver zip file.

apt-get is used to install Google Chrome.

The pip command is used to upgrade pip to the latest version and then install the requirements specified in requirements.txt.

Finally, the Docker container is instructed to run the main.py file using Python.

A big problem was the webdrive chaches and options which used a lot of storage and my solution was to restart the container every 8 hours

Logs!

Now we are at the docker side we can easily push our logs to Graylog server with Gelf:

logging:
  driver: gelf
  options:
    gelf-address: tcp://172.20.20.20:12201
    tag: surfer

The configuration specifies the following options:

gelf-address: This specifies the address of the remote server to which logs should be sent. In this case, the server is located at IP address 172.20.20.20 and is listening on port 12201.
tag: This option adds a tag to the logs to identify them. In this case, the tag is set to surfer.

Overall, this configuration file is telling the logging system to send logs to a remote server using the GELF format and to include a tag in the logs to identify them.

Prometheus!

Prometheus is a popular monitoring and alerting toolkit that is used to collect, process, and display metrics from various systems. In this code, several types of Prometheus metrics are defined, including Counters, Gauges, and Histograms, to track the flow and timing of different steps in a process.

let’s create a file named prometheus.py

from prometheus_client import Summary, Counter, Histogram, Info, Gauge
from prometheus_client import start_http_server, Info, CollectorRegistry, Metric
import os

REG = CollectorRegistry()
# Create a metric to track time spent and requests made.
SURF_COUNT = Counter('surf_count', "Surf Flow Count", registry=REG)
SUCCESS_COUNT = Counter('surfer_success_count', "Failed Flow Count", registry=REG)
VERSION = Info('surfer_info', 'surfer info', registry=REG)

choose_address_time = Gauge('choose_address_time', 'Time spent processing request', registry=REG)
choose_vendor_time = Gauge('choose_vendor_time', 'Time spent processing request', registry=REG)
fill_basket_time = Gauge('fill_basket_time', 'Time spent processing request', registry=REG)
go_to_checkout_time = Gauge('go_to_checkout_time', 'Time spent processing request', registry=REG)
go_to_payment_time = Gauge('go_to_payment_time', 'Time spent processing request', registry=REG)
go_to_payment_gateway_time = Gauge('go_to_payment_gateway', 'Time spent processing request', registry=REG)
cancel_order_payment_time = Gauge('cancel_order_payment_time', 'Time spent processing request', registry=REG)
full_time = Gauge('full_time', 'Time spent processing request', registry=REG)

# Export Google Chrome version
write_gcv_to_file = os.system("google-chrome --no-sandbox --version > /tmp/gcv.txt")
with open("/tmp/gcv.txt", "r") as f:
    data = f.read()

VERSION.info({
    "version": "0.0.2",
    "running_by": f"{data.strip()}",
})

The VERSION Info metric is used to provide information about the version of Google Chrome being used, as well as the author, the running user, and the last commit.

Overall, this code is useful for monitoring and analyzing performance metrics of a web scraping or automation process.

Main python file

and at least, we use our actions and

def add_order():
    surfer = Surfer()
    times = surfer.add_an_order()
    # Set times
    choose_address_time.set(times['choose_address_time'])
    choose_vendor_time.set(times['choose_vendor_time'])
    fill_basket_time.set(times['fill_basket_time'])
    go_to_checkout_time.set(times['go_to_checkout_time'])
    go_to_payment_time.set(times['go_to_payment_time'])
    go_to_payment_gateway_time.set(times['go_to_payment_gateway_time'])
    cancel_order_payment_time.set(times['cancel_order_payment_time'])
    full_time.set(times['full_time'])


def main():
    start_http_server(9008, registry=REG)
    i = 1
    logger.info(f"Starting the {i} Job")
    while True:
        try:
            SURF_COUNT.inc(1)
            add_order()
        except Exception as e:
            logger.critical(f"{e}")
            logger.critical(f"Can't Run the {i} Job, Try again!")
            pass
        else:
            i += 1
            SUCCESS_COUNT.inc(1)
            logger.info(f"Sleeping {RUN_INTERVAL} Seconds...")
            time.sleep(RUN_INTERVAL)

The add_order() function uses an instance of the Surfer class to add an order and returns a dictionary of times spent on each step of the process. The times are then set for the corresponding gauges using the set() method.

The main() function starts a Prometheus HTTP server on port 9008 and runs an infinite loop to continuously add orders using the add_order() function. The loop also increments the SURF_COUNT counter by 1 for each iteration, and increments the SUCCESS_COUNT counter by 1 if the add_order() function runs successfully. If an exception is caught during the execution of the add_order() function, the exception is logged and the loop continues. The loop sleeps for RUN_INTERVAL seconds between each iteration.

as you see, It will create a HTTP server and show it metrics, so add it as a target to your Prometheus scrape config like:

  - job_name: 'surfer'
    metrics_path: '/metrics'
    static_configs:
      - targets:
          - "localhost:9008"

In this case, the job name is surfer, and the metrics path is set to /metrics. The static_configs section specifies the targets where the metrics can be accessed. In this case, there is only one target specified with the IP address 172.30.30.40 and the port 9008, which is where the Prometheus server can scrape the metrics.

Grafana!

Grafana is a multi-platform open-source analytics and interactive visualization web application. It provides charts, graphs, and alerts for the web when connected to supported data sources. Wikipedia

I created a dashboard for my metrics like:

these are surfing times and counts, which helped me to find bugs and slowness in every part of the action

Screenshots can capture the state of the user interface at a particular moment in time, and can be useful in identifying visual issues or inconsistencies that may not be apparent from logs alone. For example, a screenshot could help identify an issue with a UI element not rendering correctly or a layout problem.

Logs, on the other hand, can capture important information about what the software is doing behind the scenes, including error messages, stack traces, and performance metrics. Logs can help identify issues such as incorrect data input, unexpected behavior, or system crashes.

By using both screenshots and logs, developers can gain a more comprehensive understanding of the software flow and the issues that may be occurring. This can help them more quickly and effectively identify and resolve bugs.

Conclusion

In conclusion, creating an automated testing bot using Python and Selenium has been a beneficial experience. Python is a versatile and easy-to-learn programming language, making it an excellent choice for building automation tools. Selenium, a popular automation testing framework, has provided us with the necessary tools to interact with web browsers and test web applications.

By creating an automated testing bot, we were able to save time and increase the efficiency of our testing process. The bot was able to run tests more frequently and more accurately than if they were done manually. We were able to identify issues early in the development process, allowing us to make necessary fixes before they became major problems.

For those considering implementing automated testing, we recommend starting with Python and Selenium due to their popularity and ease of use. It’s important to carefully plan and prioritize tests to ensure that the most critical components of the application are tested thoroughly. Additionally, regularly reviewing and updating tests to reflect changes in the application will help maintain the effectiveness of the automated testing process.

Overall, implementing automated testing with Python and Selenium has been a valuable experience that has improved the quality and efficiency of our testing process.

Thanks for reading !