Some Thoughts on Automation: Stop Doing Manual Tasks
Do you find yourself spending too much time on repetitive manual tasks that add little to no value? Whether it’s small tasks or larger ones, manual work that needs to be repeated over and over can feel like busy work. Tasks such as moving files, deploying code across multiple systems, adding users, gathering data from the web, adding text to images, generating CSV files, uploading products to a website, posting content online, or creating content—all these can often be automated.
The Case for Automation
Many of these tasks are indeed automatable, and while there is an initial time investment required to create the automation, the long-term benefits are substantial. Automation can save you significant amounts of time, especially for tasks that are repeated frequently.
Example: Automating Image Quote Generation
Let me illustrate this with an example. I once had a project to place text from a list of 40,000 quotes onto images. Doing this manually would take about 10 minutes per quote, totaling approximately 400,000 minutes or 277 days if you worked 24 hours a day. Clearly, this wasn’t feasible.
Instead, I designed a Python program to automate the process. Using a pool of images, the program generated roughly two quote images per second. Within about 6 hours, I had all 40,000 quote images generated. This automation reduced the time from 277 days to just 6 hours—a significant improvement.
To further enhance the process, I redesigned the system to use RabbitMQ for message passing and queuing. This allowed for adding multiple nodes (or minions) to distribute the workload. With two nodes, the time dropped to 3 hours. Adding a third node reduced it further to 2 hours. While adding more nodes could reduce the time even more, there is a point of diminishing returns.
Example: Automating Content Upload to WordPress
After generating the content, the next step was to upload the quote images and their corresponding text to a website. Manually creating 40,000 posts would be an enormous task. However, with planning, I ensured each quote image was assigned a unique ID that corresponded to the quote text used to create it. This allowed for efficient mapping of the image to its text.
Using this information, I generated templates and saved them in groups to a CSV file. This CSV file included useful information such as titles, tags, and categories, making it easy to import the content into WordPress. The final step was to import the CSV content into WordPress, and just like that, 40,000 posts were created effortlessly.
Conclusion
The key takeaway is that if you have a task that is repetitive and time-consuming, consider automating it. Automation can turn what seems like an insurmountable workload into a manageable process, saving you time and effort. With the right planning and tools, you can streamline your workflows, improve efficiency, and focus on tasks that add real value. Let machines handle the repetitive work, so you can concentrate on more important things.
Have a Creative or Technical Project in Mind?
Looking for guidance, insights, or a fresh perspective on your technical or creative journey? Or just somebody to chat with?
Reach Out
jamie@jamiestarling.com