Home Automation Series: Weather Data


This is probably over kill, but I’ve always had a fascination with the weather, and so one of the things I wanted to do was hold an enormous about of weather data, so I could write logic to analyze it and provide me with data about unusual trends, or possible up coming weather.

I read through several different Weather API providers and settled on DarkSky, who had two important factors: History all the way back to 1937 (for the most part) and cheap rates for excessive API calls.

Example output from my RestAPI

Using DarkSky, which has their own engine to “predict future weather”, I’ve populated weather data all the way through 2024. Every day though, my code will reach out and pull down accurate weather for each day. Ensuring valid data.
Right now, this process is dependent entirely on the internet, in the future though all weather data will be collected from a locally hosted weather station.

Why all the data though?

A major part of this will be a digital assistant, which I will probably call Jarvis. With that, I want to be able to rather quickly calculate weather records and have Jarvis be able to provide me with weather from a specific day or factoids about weather. Ideally, I think it would be pretty neat to write some code that could try to predict what the weather will be, then use actual weather recordings to prove or disprove (validating the learning process). This way, Jarvis could inform me of possible weather pattern changes or be able to tell when upcoming days will be sunny or not.

The real reason for the data hoarding though, is primarily because DarkSky was bought by Apple and it’s rumored the API will go away at some point. Which means, my ability to write code that will be dependent on that API is in question. So, instead, I’ve captured all the data I need prior to the API going away.


I’ll eventually look at plotting out all of the data in a scientific format using Matplotlib, this goes back to me being fascinated by weather data. It would be pretty neat to see how often trends repeat in terms of temperature spikes and dips, how often we actually get snow accumulation, or monthly average rain fall. I’ve always kind of been a bit of a data scientist, I’ve just never had the ability to act on the desire.

All of this data will feed into the logic that will be used to adjust the house temperature using the HVAC controls I’ve been building.

Thanks for hanging around!

-Dan

Home Automation Project


For as long as I can remember, I’ve always thought the idea of having automated controls was pretty cool. This became especially pronounced after watching the Iron Man movies. The idea of have something like Jarvis in control things was really cool and I really wanted to create something like it.

In 2011, I started watching a guy on YouTube who had taken a MacMini and wrote a bunch of AppleScripts to create what he was calling Jarvis. He was sharing his ideas and features and for the first time I thought it might be possible to create such a thing without needing to spend a ton of money to buy it.

I originally borrowed his Library.scpt file and used many of it’s pre-defined functions in my code. This worked well until many of things it used were disconnected by their vendor, like Google Latitude, and Yahoo Weather’s API. I spent a long time writing AppleScript to do stuff like, answer questions using Wolfram Alpha and check my Calendar for events, which was cool. But it wasn’t “useful” to me as much as I wanted it to be. I stopped writing AppleScript somewhere around 2015 and the project sat idle for the next 5 years, occasionally I would poke at it.

Fast forward to 2019, when I started learning Python. I had known about Python for a long time but was hesitant to learn it because of the learning curve. My learning style is very different than other peoples, I learn by doing. My first foray into Python was a project for work, a utility that would read over files from one of our products and build a database of objects and cross-reference several data points (both dynamically and statically). After a year of working on that project, it had received many positive reviews and was adopted into a new product. Not bad for a first go, eh?

With this new found Python knowledge, it reignited the idea of “Jarvis”. Could I create such a thing with Python? There are so many videos on YouTube where people have created really simple single function “bots” and have called them “Jarvis”, stuff like, “Call me Dan” and the bot responds, “Hi, Dan”. Don’t get me wrong, if you’ve never written code before, that’s a big accomplishment. I wanted something much more advanced.

I started writing a bunch of different functions to do stuff like collect weather data and send tweets, which again were somewhat useful in the grand scheme of things. After sometime I decided I’d use those as parts of a bigger picture, and started focusing on what I needed/wanted this thing to do when done.

I knew it needed to:

  • Be aware of external temperature data
  • Be able to detect temperature data from multiple rooms in the house
  • Be able to adjust both the thermostat and the various dampers in the house to better balance temperature
  • Notify me of things like unusually high humidity in a room, or room temperature that will not adjust.
  • Automatically control lights (both internal and external), either by voice command or time of day.

Now, much of this is already possible with Google Assistant, Amazon Alexa or Siri, but I don’t want to be bound by what the limitations are for those. I want to be able to build in fail-overs for things. Like, if a room temperature won’t adjust, close all other dampers by some small percentage for a set time and turn the fan on, forcing more air into that room, and thus more air out of that room. I don’t know that I could have that kind of custom workflow with the others.

With all that being said, how will this work?
I’ve built a docker container that will run Python + Flask for the front-end. This way, I can have a TV with system data displayed, room temperatures, damper percentage, outdoor temperature, thermostat schedule and some other bits of data. Within this docker container will be several pieces, the first being a scheduler. The scheduler will automatically trigger several operations, like collecting all temperature data once an hour, or doing a systems check and sending me a push notification if somethings wrong.

Second, will be the vocal side, I haven’t yet decided how I want to go about communicating by voice, but there is code to handle that. I’ve thought about having a bluetooth that I wear, but that seems tedious. I’ll revisit that a bit more in the future.

Third, a Rest API. Why have a Rest API on this? Integrations. One of the things I thought about was the ability to have “outside” things integrate into my system. Things like purpose built Raspberry Pi’s that can either be collecting data or pulling data from the system and doing something. The primary purpose here is HVAC damper control. There will be a couple of Raspberry Pi’s around the house with equipment wired to dampers in our HVAC ducting, when the main system I’ve written needs to check temperatures in a room and adjust the damper, it will send a POST command to the appropriate Pi to adjust the damper, and the damper will make the change and return data points about damper position among other things. The main system can then note the change and it will now have in a database, the damper value.
Another piece to this is having this entire system on its own private network that is on battery backup.

Thanks for reading.. More to come.

-Dan

Farewell 2020…


2020 has gotten a pretty bad reputation. It’s the year the pandemic ruled the world and lockdowns occurred. However, something I think a lot of people don’t see is that, 2020 gave us the ability to grow closer to our family and learn to adapt and overcome. But many people won’t see it that way, they’ll see the negative side before they see the benefits. They’ll see that their ‘right’ to go into a restaurant and eat was taken away, or the their ‘right’ to be in a large group of people was taken away.

What’s interesting to me is everybody seems concerned with their ‘rights’ but not so much about ensuring the people around them are healthy and don’t die. They’d rather enjoy themselves than be safe.

I’m hopeful as we move into a new year, people become more open minded and see benefits before negativity.

Hello 2021. Let’s rock this thing!

Code Junkie


In the last year, I’ve spent a bit of time learning and building with Python. So much so, that I’ve actually written a fully fleshed out application for work that makes short work of what used to take a long time manually. I’ve been finding every excuse to write Python code. Video download/processing? Sure. Rewriting my Jarvis to use both Applescript and Python, why not? I’m getting there though. (more details here: https://drwalker.wordpress.com/2012/09/07/jarvis/)

Python is really a fun language to write with. I’ll post some more project information as time goes by.

Adding on to backup bash scripts


So, ages ago, I wrote up a couple of bash scripts that automated the backup process on a server. It’s all text based, so you’d have to be logged into a terminal or be ssh’d into the box.

Today, I got to thinking, I should give it a pretty web gui that makes the process easier and functions through a web gui.

It’s still really early in the through process, but I’m intrigued by the project and will start working on it soon.

Original Post: http://www.merval.org/2012/03/bash-scripts/

Github: https://github.com/merval/BackupScripts