Bayesian linear regression in Python to quantify my sleeping time
Over the years, I have heard many of my friends comment- Do you ever sleep? Well, I won’t blame them. Generally, I am the first one up even after a late night, although that has changed over time. In my early 20s I used to sleep for 5 or 5.5 hours(We would catch a late-night movie, reach home by 2 am and still reach the office by 8 am and be functional) but nowadays it’s between 6 and 6.5. Of course, there are days when I linger in bed for much longer because the energy dissipates when you get older. …
Satellite data analysis in Jupyter to find out how green London is!
I have been interacting with a potential oil-gas-energy client for whom I developed a proof of concept on using sound data at the oil rig sites. I have another meeting next week in which use cases around satellite imagery data will be discussed with the urban planning department of the firm. I want to learn the basics of GIS data analysis programmatically so that I am not exactly a neophyte by next week and come up with a business use case that’s relevant to the urban planning department. Thus, I have been in pursuit of a python library that’s powerful and strong enough to accomplish all the complicated tasks of remote sensing. …
I had one day to learn the basics of Satellite Imagery Analysis using Python
Things I want to accomplish in one day:
1. Learn how to download the data from free resources
2. What python libraries are available for satellite image analysis
3. Get acquainted with the vocabulary
4. Run a real-life scenario on the data downloaded. In this case, we will check the green cover in Greenland using NDVI.
I have been interacting with a potential oil-gas-energy client for whom I developed a proof of concept on using sound data at the oil rig sites. I have another meeting next week in which use cases around satellite imagery data will be discussed. …
A gentle introduction to price elasticity of demand(PED) in Python
Where were you in 2016–2017? I was in NYC — the city that starts the trends from fashion to food and from entertainment to music. I joined the wagon and participated in a few but one that has stuck with me is avocado in the form of avocado toast or guacamole.
The year 2017 saw an upsurge in the prices of the fruit with avocado scene booming exorbitantly; I have paid as high as $3.65 for a single Hass avocado at local Whole Foods. There is a joke that all millennials are poor and live with their parents because they spend all the money on avocado toasts. For some reason, the fruit is considered premium and is marked at a higher price; well, one of the reason is demand. …
Drinks, an informal conversation, a few graphs helped someone view the data in a new light.
The photograph isn’t pretty when compared to what people post in their blogs but this old piece of paper has an interesting story associated with it.
A few days ago, I was cleaning my bag which has too many compartments — typical of Swiss gear and I found a pocket that was not touched for years. A small hidden pocket acting as home for business cards, crumpled papers, and receipts from the past. That’s when I discovered the old, stained paper with this crude and rough sketch of what I scribbled down around 4 years ago.
Spatial info: An Irish bar in Denver…
Making Sense of Big Data
I translated a scientific paper into python to check whether it can help in reducing space and time consumption
I was (am still) running a few deep learning networks for face detection on my humble machine but I kept bumping into memory and CPU/GPU consumption issues. I can either reduce the number of images I am training or find a way to make my algorithm scale on my machine.
You can say, why not rent a high-end machine or buy some server time! Well, both these options are legit but maybe there is a way to reduce high dimensional image data or maybe not and I wanted to experiment a little before I shift direction.
It is relatively old and much used(in fact overused) piece of literature but in any case, I wanted to implement it(from scratch) https://www.mitpressjournals.org/doi/pdf/10.1162/jocn.1991.3.1.71 …
I had 1 night to decide whether spending a large amount of time on Graph database will be fruitful.
I spent a large amount of time last year on developing a recommendation system for a Telecom client’s users; It turned out to be a massively difficult problem to undertake and accomplish in a short stipulated time. I was faced with a similar-sized problem last week and had a short quick around time for devising an initial strategy. I was well aware of the landmines that the data-driven methods have so I wanted to test another approach.
Last year, someone had mentioned Neo4j DB for Recommendation System but I didn’t pay much heed to it. I heard about Neo4j for the first time in 2016 when I downloaded it along with the Panama Papers data to expose the shameless tax avoiders and owners of the overseas shelf companies from my country. I ran a query here and there for an hour and a half while sitting at Cafe Jax on 84th St. …
Let’s dig into Spotify and understand ourselves more with help of Python
Rekindling the spirit from the old post, I continue down the rabbit hole in identifying the characteristics of the songs that I prefer.
Essentially, while working I listen to a few songs and this one was streamed 221 times last year as my Spotify’s 2020 Unwrapped told me.
After an initial audio profiling and spectrogram analysis, I could gauge that I prefer lower frequencies songs, beats in 115 to 130 range, and bright timbre but that’s all I could understand and I wanted more qualitative information.
So, I decided to exploit the developer’s API of Spotify and registered as a developer at https://developer.spotify.com/dashboard/applications. You can create any dummy app, give it whatever name you want to and Spotify will spit out a client Id and client secret key for you. …
A deeper understanding of the audio data to understand taste in music
A friend of mine and I share the annual ritual of comparing our ‘year’ unwrapped data that is sent religiously by Spotify every year. We compare the number of minutes, new artists, new genres, most streamed song, most dominating genre and whatnot. This year was no different but this year I finally decided to wrap this project that I intended to finish for years.
Spotify has indeed changed the way I listen to music and has made me realise that there are certain beats that I gravitate to and the last few days were spent in the pursuit of understanding what kind of music do I prefer and what are those special undertones in certain songs that make me stream them 221 times a year. The other goal was to find some similar songs to the one that I am after. You can argue that one can rely on products that are already there for such purposes i.e. song recommenders, beat identifier etc but the point is to make myself suffer through an inordinate amount of python code and eventually gain first-hand insights that I would inject in conversations with friends and strangers to sound extra douchey. …
Written by @prashantmdgl9 who is not an epidemiologist.
We are living amidst a pandemic.
It’s common knowledge and not an astute observation that every other article in the newspaper or websites is about COVID-19; there are loads of messages circulated every day on social media on how to prevent the crisis, conspiracy theories, people claiming to have found the cure, and meticulously orchestrated attempts to link the virus with every other thing in the universe.
One such cohort of articles that suddenly proliferated and conspicuously stands out belong to data science. These are easy to spot as they are embellished with multifarious graphs, visualisations that are based on mined datasets and eventually they make a blanket conclusions such as it is just another flu, no need of mass hysteria or worst is yet to come. …
About
