NAIMbot - Aim assistant based on machine learning

I essentially created an aim assistant for a game by using machine learning which I would later call NAIMbot.
Purely with the visual input of a screenshot, NAIMbot is able to detect an enemy on screen and calculate the distance from the center of the screen to the enemy and accurately aim for the target.

▬▬▬▬ Beginning ▬▬▬▬
My first obstacle was how to move a player and the mouse externally. After a lot of research, it turns out, DirectX applications need special input and it was essential to get this step right since everything is built on top of this.
Next step was trying to make calculated input look human. For this I needed to create a function to reconstruct parabolas, so that the beginning of the the mouse movement was slower than the middle since a human needs to accelerate the physical mouse.

After perfecting that, I went on to create recoil compensation for each weapon by using computer vision and a GIF that contains all locations of the bullets.
This leads to the first interesting part of the project.

▬▬▬▬ Computer Vision ▬▬▬▬
In order to to recognize the current weapon or which side one is currently on, NAIMbot needs to read it from somewhere on screen, just like a human would.
I used cv2 to solve this problem. cv2 is a Python library focused on computer vision. With it I was able to automate data collection and process it.

▬▬▬▬ Data Collection ▬▬▬▬
While usually data has to be hand labelled, I have found many ways to automate data collection.
I unfortunately cannot disclose too many details, since this is my competitive advantage but in the case of this project, I used a mixture of in-game data and external memory data, to collect data. Of course that data needed processing which was done by using cv2 and knowledge of neural networks.

▬▬▬▬ Machine Learning ▬▬▬▬
The machine learning part boils down to a classification part and a localization part. For localization I used a region convolutional neural network. Since I wasn't happy with the accuracy and speed, I switched to Google's object detection since it has entire research teams behind it.
The speed and accuracy were finally reasonable. But to improve it further, I adjusted the hyper parameters and altered the image input size.
The difficult part was to balance speed and accuracy since it is very dependent on the hardware used.

▬▬▬▬ Other ▬▬▬▬
There are a few more aspects to this project, since my original plan was to make NAIMbot commercially available, like connecting a database to a website and writing to it each time a purchase has been made (to circumvent the hassle of using licenses for software) or the creation of a graphical user interface. These things were more straightforward so theres no need to dive deeper but if you would like to know more about anything, feel free to ask away!

To conclude:
What this means is, I created a non-intrusive aim assistant, which has never been done before. It opens up many doors for the competitive scene where aim assistance is very popular.
This is why I have halted the project. I do not want to be responsible for upsetting the entire competitive scene which I do enjoy myself.
Instead, I'd like to use this project as experience. The fact that I have gained an immeasurable amount of knowledge from this project will be used to help others with their problems and ideas.