As already mentioned in other chapters, I already immersed myself in programming and the development of algorithms during my bachelor’s degree, which continued throughout my master’s degree and beyond. As a result, I have acquired strong skills in the acquisition, processing, refinement and analysis of data. Depending on the task, I am able to prepare, restructure, standardize, visualize and integrate data into a process.
Projects I have successfully worked on include data extraction, financial data, audio and video data, social media, text analysis, email processing and game theory research.
Example online data
The Internet could be described as the greatest treasure trove of data that has ever existed. And it can be accessed free of charge.
There are automated techniques for making Internet-based data accessible in a targeted manner, such as web scraping. This is a process with which internet-based data can be extracted and then used and analyzed.
Another option is to process the data obtained dynamically in an online application. Have you ever searched for flights on a platform that summarizes the offers of all airlines? This is a typical use case.
One of my projects, for example, was to extract the participant data from the CAMX trade fair and transfer it to a separate table. This then contained all the relevant company data.
Example Bots
Bots are programs that can be used to search networks in order to collect targeted data of interest. If you are now thinking of search engines, you are not wrong. In fact, search engines also work via such bots. For example, there is a Google bot that searches the internet to make it findable through user queries. I have written bots that work in a similar way to such search engines or bots that process content from social media.
Post processing
Depending on the type of data collected, post-processing may be useful or necessary.
For example, data can be mapped to visualize locations. This enables or simplifies the evaluation of data with geographical relevance.
I have published a couple of my projects on my GitHub-Profile.