A Data Scraping Training session was hosted by MCRIT Multicriteria Planning, Work Package 2 (Co-design of data collection approaches, databases, and RUSTIK information system) leaders on 29 April, 2024. This training was tailored to meet the specific needs of Living Labs involved in the data experimentation process.
What is Data Scraping?
Data scraping, also known as web scraping, is the automated process of extracting information from websites or databases. This enables us to efficiently gather large volumes of data for analysis, uncovering valuable insights and trends.
Training Highlights
- From Data to Geospatial Information: Participants learned how to convert raw data into meaningful geospatial information. This process is crucial for visualizing and understanding the spatial dimensions of the data collected.
- Relevance of Geographical Information for RUSTIK: This part of the session highlighted the importance of geospatial analysis. Moreover, participants gained insights into territorial dynamics and learned how geospatial data facilitates informed decision-making in various domains such as urban planning, environmental management, and transportation logistics.
Scraping Techniques
Attendees discovered methods to extract data from various online sources, automating the process for streamlined workflows and access to up-to-date information. Tools such as Apify and Geofabrik were introduced.
- Apify: A platform that allows users to automate web scraping tasks without writing any code, making it accessible to users of all skill levels.
- Geofabrik: Provides geospatial data extracts and tools for OpenStreetMap, enabling users to access and process geographic data for a wide range of applications.
- Transforming Scraped Data into Useful Information for Living Labs: the training focused on how to process and transform the data extracted from websites into actionable insights for the Living Labs‘ experimentation processes. Participants learned best practices for cleaning, structuring, and analysing the data to support their specific needs and objectives.
To conclude, as the training wraped up, it’s time to put the knowledge into action. Our Living Labs partners will leverage these tools for their data experiments. Stay tuned and follow the project to catch the exciting results!