Deep Reflections

Close to the end of A Swim in a Pond in the Rain, George Saunders looks back on a class from his graduate school. His professor, a short-story writer, gave a reading of Chekhov. Saunders didn’t know much about Chekhov then: he thought Chekhov’s stories were “mild” and “voiceless”. The professor reads three stories, and in doing so, brings Chekhov to life. After that reading, Saunders was not only convinced of the power of the short story, but also “desperate” about writing better short stories himself. [Read More]

The Sense of an Ending is a slim novel by Julian Barnes. I don’t have much to say about the story, but I was drawn to the main character, Tony Webster.

[Read More]

We are all familiar with sci-fi stories of evil robotic overlords. I’m not into science fiction as a genre, yet I thoroughly liked the movies Ex Machina (2015) and The Matrix (1999). Both movies are nominally about artificial intelligence, but ask deep philosophical questions.

[Read More]

I recently read Ghachar Ghochar, a novella by Vivek Shanbhag. The original is in Kannada, but I read the English translation by Srinath Perur.

The novel is slim; it runs to a little more than 100 pages. This is significant, because within these pages, the author still tells a wonderful story. It’s a story that leaves a lot unsaid, for the reader to fill in the details.

[Read More]

A couple posts ago, I described the machine learning model that we developed to predict the extent of cropland irrigation worldwide. This post is a collection of all the things I learned about Google Earth Engine that went into running a global-scale model successfully.

[Read More]

It was the first class of a data science course in the MIDS program. Our instructor started off with a question: “Do you think this academic program gives you a better career?” We gave a bunch of answers, quite the ignorant folks that we were. Of course, is this even a question at this point?! Yes, here are all the new things we’re learning. Yes, the instructors are top-notch. Yes, look at the syllabus and the projects, yada yada … nothing surprising there.

The instructor asked, “Yes, but how do you know that the program is making you better? What if you joined the program because you were motivated enough to apply and work through it, and therefore you’d get better in your career anyway?”

[Read More]

For our capstone project at UC Berkeley, we worked with the Department of Environmental Studies. We developed a machine learning model that we continued to improve even after the course term, and I’m happy to report that it is now under discussion to be a published paper – a first for the program itself, according to our lecturer.

Update Aug/5/2020: New blog post: a behind the scenes look on how we built the model.

[Read More]

There are so many great ideas in engineering we can take home and apply to our own lives. Today I will talk about one of them: lazy evaluation.

[Read More]

We live in a “knowledge economy”. Every day brings with it new information. Do you have an external system to store your knowledge efficiently and effectively?

[Read More]

This post contains various notes from the second half of this year. It was a lot of learning trying to get a streaming model working and ready in production. We used Spark Structured Streaming, and wrote the code in Scala. Our model was stateful. Our source and sink were both Kafka.

[Read More]