In today’s competitive world, success depends heavily on how fast you reach the market and capture it. This is one of many significant reasons behind the success of the IT giants like Amazon, who release their code every 11.7 seconds. With the advent of agile methodology, the speed of product and feature delivery has grown manifold. Since it is impossible to do frequent releases with multiple manual touchpoints in the development and QA process, automation becomes necessary.
Read More about Salesforce Commerce Cloud with Azure DevOps
For DevOps, automation is the means to develop and deploy software builds faster while adhering to quality standards. It reduces the friction between key interrelated tasks and ensures their proper coordination. Automation amplifies the performance of individuals by taking over repetitive tasks. Many companies have adopted DevOps automation and many more are likely to follow in the future.
Read how you can develop a custom exporter for Prometheus using Python.
Automation scripts written in Python language can help in DevOps automation. As a language, Python has several features that enable automation. It has an Ideal guideline, known as ‘PEP 20,’ for test automation. Pytest, a test framework in Python, is one of the most popular test frameworks in all languages.
Let's go through a few Python modules for writing automation scripts:
1. Requests
Python Requests allows you to send HTTP requests. Using this module, we can post or retrieve the data from a Rest API. There are many methods included in this module, like:
- GET
- POST
- PUT
- DELETE
All these methods perform particular actions like adding a comment (PUT), retrieving data (GET), or deleting a user field (DELETE).
Example: r = requests.get(“url”,auth=('username', 'passwd'))
In the above example, we used the Python Requests module to get the information from the URL provided. You can get information like status code, JSON data using status code() and json() methods, respectively.
2. Paramiko
Python Paramiko allows users to log in to a virtual machine’s server using automation scripts. It enables the users to perform ad-hoc commands by using SSH client. All commands get executed in the connected remote machines.
Example - ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(ip, username=user, password=passwd)
stdin, stdout, stderr = ssh.exec_command("command")
out = stdout.readlines()
In this example, we connected to a server using its I.P., username, and password. ‘stdin’ stores the input passed, ‘stdout’ stores the output obtained, and ‘stderr’ stores the error that occurred.
3. Pandas
Pandas module is for data manipulation. It is the most useful module for data science engineers. But in DevOps, Pandas helps to store the massive dump of data into CSV or Excel files within seconds.
Example - pandas.read_json("file.json").to_excel("output.xlsx")
In the above example, we used the read_json() method to read a JSON file and migrated its data into an Excel file.
4. Selenium
One of the most popular modules used in Python to perform testing and connect to different browsers. By using Selenium with Python, we can hit a URL and perform actions like clicking on a button. We can even use other sub-modules like ‘Pyautogui’ with Selenium for different purposes, like filling some text fields.
Example - from selenium import webdriver
driver = webdriver.Chrome()
driver.get("https://www.gspann.com")
In the above example, we have used Chrome WebDriver to access the Chrome browser. If you want to use this module for other browsers, you need to use different WebDrivers accordingly.
5. Beautiful Soup
Beautiful Soup in Python is the most useful module for pulling data from HTML and XML files. To parse data from the content in HTML and XML format, we need to create a BeautifulSoup object for it. We can then use the BeautifulSoup methods on the soup object that we created for the URL. We can even apply many filters using the tags on the obtained data content, like soup.find_all(“< tag>”).
Example - URL = “https://www.gspann.com”
content = urllib2.urlopen(url).read()
soup = BeautifulSoup(content)
In this example, we are extracting all the content of the page into a soup variable. Using the soup variable, we can obtain the information of a specific tag by methods like find_all.
6. OS
The operating system in Python is a basic module. It performs ad-hoc commands on the base OS, which we are executing. All commands run on the current machine.
Example - os.system(“dir”)
In the above example, ‘dir’ gives an output of the list of directories. They are many other similar methods like os.remove(), os.rename(), os.close(), etc.
7. JSON
JSON is the syntax for storing data in the form of dictionaries and lists. Python JSON is a built-in module that can parse the JSON data and even convert the other type of data into JSON format.
Example -
x = {
[“data”,”stored”]
}
json.dump(x)
In this example, we can observe that x is a dictionary that is holding a list containing elements. We are using the dump function to parse the data.
8. XLWT
This module helps to create an Excel file and store data after customizing it according to the cells. It is a good module to work with Excel files, but it takes more time when compared to Pandas. This module allows the user to define the logic according to the requirement, like filling data only in a particular cell.
Example -
x=Workbook()
sheet=x.add_sheet()
sheet.write()
In the above example, we initialized an Excel workbook in a variable and started performing actions like adding a sheet and writing data into that sheet using that variable.
Python helps us in automating repetitive things and it offers a lot of flexibility. It has a small learning curve compared to other programming languages. However, it all depends on how much time and effort a DevOps engineer spends on aptly utilizing its different modules, methods, and frameworks to enable automation. It will be apt to say that Python is core to DevOps automation and mastering this language is necessary to enable DevOps.
Anurag Bhargav
Associate Software Engineer
Published Jan 03 2020
