Address
304 North Cardinal St.
Dorchester Center, MA 02124
Work Hours
Monday to Friday: 7AM - 7PM
Weekend: 10AM - 5PM
I finally upgraded one my my repositories using Python 2to3, and I wanted to share how.
Python 2to3 – Introduction
First, if you were not aware, Python 2 is officially end-of-life.
While I am a little late to the update party, I finally got it done on my larger repository.
There are still repositories and libraries out there that still run on Python 2.7, but hopefully they will all phase out soon.
Note that you should already have the tool installed, as Python released it back in 2018.
Usage
Usage of the tool is simple, and you can see all the flags below.
root@kali:~/tools/SecurityTools$ 2to3- --help Usage: 2to3 [options] file|dir ... Options: -h, --help show this help message and exit -d, --doctests_only Fix up doctests only -f FIX, --fix=FIX Each FIX specifies a transformation; default: all -j PROCESSES, --processes=PROCESSES Run 2to3 concurrently -x NOFIX, --nofix=NOFIX Prevent a transformation from being run -l, --list-fixes List available transformations -p, --print-function Modify the grammar so that print() is a function -v, --verbose More verbose logging --no-diffs Don't show diffs of the refactoring -w, --write Write back modified files -n, --nobackups Don't write backups for modified files -o OUTPUT_DIR, --output-dir=OUTPUT_DIR Put output files in this directory instead of overwriting the input files. Requires -n. -W, --write-unchanged-files Also write files even if no changes were required (useful with --output-dir); implies -w. --add-suffix=ADD_SUFFIX Append this string to all output filenames. Requires -n if non-empty. ex: --add-suffix='3' will generate .py3 files.
I prefer to run the program recursively, and not automatically write any of the changes.
That said, the -w flag is great if you have a much larger code base with good backups!
Performing My Updates
I wanted to update my SecurityTools repository, as this is where most of my current code lives.
First, I ran the tool on my entire SecurityTools repository. I wrote most of my tools in Python, so I figured that there would be plenty of changes to make.
As you can see, most of the changes were related to the Python 3 print function.
root@kali:~/tools/SecurityTools$ 2to3- . RefactoringTool: Skipping implicit fixer: buffer RefactoringTool: Skipping implicit fixer: idioms RefactoringTool: Skipping implicit fixer: set_literal RefactoringTool: Skipping implicit fixer: ws_comma RefactoringTool: Refactored ./AlexaPortScanner/alexaPortScanner.py --- ./AlexaPortScanner/alexaPortScanner.py (original) +++ ./AlexaPortScanner/alexaPortScanner.py (refactored) @@ -1,4 +1,4 @@ -from __future__ import print_function + import socket import sys RefactoringTool: Refactored ./BinToHex/binToHex.py --- ./BinToHex/binToHex.py (original) +++ ./BinToHex/binToHex.py (refactored) @@ -14,4 +14,4 @@ ctr = 0 ctr += 1 shellcode += "\"" - print shellcode + print(shellcode) RefactoringTool: Refactored ./BurpVERBalyzer/VERBalyzer.py --- ./BurpVERBalyzer/VERBalyzer.py (original) +++ ./BurpVERBalyzer/VERBalyzer.py (refactored) @@ -25,7 +25,7 @@ from org.python.core.util import StringUtil import string except ImportError: - print "Failed to load dependencies." + print("Failed to load dependencies.") VERSION = "1.0" callbacks = None @@ -75,7 +75,7 @@ callbacks.registerScannerInsertionPointProvider(self) callbacks.registerScannerCheck(self) - print "Successfully loaded VERBalyzer v" + VERSION + print("Successfully loaded VERBalyzer v" + VERSION) return # helper method to search a response for occurrences of a literal match string ... <snip> ... RefactoringTool: Files that need to be modified: RefactoringTool: ./AlexaPortScanner/alexaPortScanner.py RefactoringTool: ./BinToHex/binToHex.py RefactoringTool: ./BurpVERBalyzer/VERBalyzer.py RefactoringTool: ./DNSRickroll/dnsRickroll.py RefactoringTool: ./ECBPlaintextAttack/ecbAttack.py RefactoringTool: ./ECBPlaintextAttack/ecbServer.py RefactoringTool: ./FileIntegrity/fileintegrity.py RefactoringTool: ./IpExpander/ipExpander.py RefactoringTool: ./PortScanner/portScanner.py RefactoringTool: ./PyDHCPDiscover/dhcpdiscover.py RefactoringTool: ./PythonShellcode/pythonShellcode.py RefactoringTool: ./RSAGenKey/genKey.py RefactoringTool: ./ReverseShell/reverseShell.py RefactoringTool: ./ZipCracker/zipCracker.py
The only file that had other changes was my old file integrity checker.
root@kali:~/tools/SecurityTools$ 2to3- ./FileIntegrity/fileintegrity.py RefactoringTool: Skipping implicit fixer: buffer RefactoringTool: Skipping implicit fixer: idioms RefactoringTool: Skipping implicit fixer: set_literal RefactoringTool: Skipping implicit fixer: ws_comma RefactoringTool: Refactored ./FileIntegrity/fileintegrity.py --- ./FileIntegrity/fileintegrity.py (original) +++ ./FileIntegrity/fileintegrity.py (refactored) @@ -6,7 +6,7 @@ import requests import shutil import smtplib -from urlparse import urlparse +from urllib.parse import urlparse def calculateOriginalValues(fileUrl, tempFile): r = requests.get(fileUrl) @@ -21,10 +21,10 @@ calculated_md5 = hashlib.md5(f.read()).hexdigest() if originalHash == calculated_md5: - print "MD5 verified." + print("MD5 verified.") return None else: - print "MD5 verification failed!" + print("MD5 verification failed!") file1 = open(fileName, "rb").readlines() file2 = open(originalFile, "rb").readlines() @@ -44,7 +44,7 @@ try: while 1: - diff_string = diff.next() + diff_string = next(diff) if not (diff_string[0] == " "): differences += diff_string + "\r\n" if diff is None: @@ -55,7 +55,7 @@ def sendEmail(configFile, diffString, isSSL, hasAuth): config = {} - execfile(configFile, config) + exec(compile(open(configFile).read(), configFile, 'exec'), config) sender = config["sender"] recipient = config["recipient"] RefactoringTool: Files that need to be modified: RefactoringTool: ./FileIntegrity/fileintegrity.py
Other than iterators and print methods, I had to learn how to update my execfile call.
After looking at the official documentation, I updated my code to reflect the 2to3 suggestions exactly.
Committing the Changes
When I finished my changes, I went to commit the changes to my repository.
Unfortunately, I was getting an authentication error when I tried to use the git command line tool.
root@kali:~/tools/SecurityTools$ git add . root@kali:~/tools/SecurityTools$ git commit -m "Updated SecurityTools for Python3" [master b5bdff4] Updated SecurityTools for Python3 12 files changed, 66 insertions(+), 67 deletions(-) root@kali:~/tools/SecurityTools$ git push origin master Username for 'https://github.com': doyler Password for 'https://[email protected]': remote: Invalid username or password. fatal: Authentication failed for 'https://github.com/doyler/SecurityTools.git/'
I rarely used this for GitHub, so I had to poke around to see why it wasn’t working.
As it turns out, I needed to use a personal access token, as opposed to my password.
Once I fixed this, I was able to commit my changes just fine!
root@kali:~/tools/SecurityTools$ git push origin master Username for 'https://github.com': doyler Password for 'https://[email protected]': Enumerating objects: 49, done. Counting objects: 100% (49/49), done. Delta compression using up to 16 threads Compressing objects: 100% (25/25), done. Writing objects: 100% (25/25), 3.02 KiB | 1.51 MiB/s, done. Total 25 (delta 14), reused 0 (delta 0) remote: Resolving deltas: 100% (14/14), completed with 14 local objects. To https://github.com/doyler/SecurityTools.git e082b9f..b5bdff4 master -> master
As you can see, the commit went through, and I updated my repository.
Python 2to3 – Conclusion
I should have updated my repositories sooner, but this was something that I was putting off.
There is still some code in other repositories that I will need to update but let me know if you actually use/need any of it.
Finally, let me know if there are any updates that you’d like to see to any of these tools, or new tools in general!
Ray Doyle is an avid pentester/security enthusiast/beer connoisseur who has worked in IT for almost 16 years now. From building machines and the software on them, to breaking into them and tearing it all down; he’s done it all. To show for it, he has obtained an OSCE, OSCP, eCPPT, GXPN, eWPT, eWPTX, SLAE, eMAPT, Security+, ICAgile CP, ITIL v3 Foundation, and even a sabermetrics certification!
He currently serves as a Senior Staff Adversarial Engineer for Avalara, and his previous position was a Principal Penetration Testing Consultant for Secureworks.
This page contains links to products that I may receive compensation from at no additional cost to you. View my Affiliate Disclosure page here. As an Amazon Associate, I earn from qualifying purchases.
