Deepak Nagaraj

Ego is the problem

n.deepak@gmail.com (Deepak Nagaraj) — Wed, 31 Dec 2025 00:00:00 +0000

This is a blog post that’s kind of self-defeating.

Semantic Search on PDFs with Elasticsearch

n.deepak@gmail.com (Deepak Nagaraj) — Fri, 18 Apr 2025 00:00:00 +0000

This document shows semantic search on Elasticsearch on PDF documents without any additional client code.

SSO and RBAC for Elasticsearch (Kibana)

n.deepak@gmail.com (Deepak Nagaraj) — Thu, 17 Apr 2025 00:00:00 +0000

This page documents the steps for:

using an external Identity Provider for user authentication to Kibana (single sign-on, aka SSO)
providing differentiated access to Kibana based on the assigned role to the authenticated user (role-based access control, aka RBAC)

System Setup for GPU Performance Measurements

n.deepak@gmail.com (Deepak Nagaraj) — Sun, 16 Mar 2025 00:00:00 +0000

In this document, I outline the steps to set up a system to take NVIDIA GPU performance measurements. This could be required, for example, to analyze and improve the performance of an LLM.

Field Level Access Control with Elasticsearch

n.deepak@gmail.com (Deepak Nagaraj) — Sat, 15 Mar 2025 00:00:00 +0000

This page documents experiments with field-level access control in ES8.x.

Favorite Books of 2024

n.deepak@gmail.com (Deepak Nagaraj) — Tue, 31 Dec 2024 00:00:00 +0000

Some favorite books of 2024, and further thoughts they evoke.

Opinion: With Apache Beam, Tread Carefully

n.deepak@gmail.com (Deepak Nagaraj) — Tue, 25 Apr 2023 00:00:00 +0000

Machine learning teams today often want to process data in real-time. They also standardize on Python as a programming language. Apache Beam supports Python and gives a lot of hope, but my opinion is that this combination is practically too complex and unreliable to put in production.

Implementation Notes on LLMs for the Enterprise Use-Case

n.deepak@gmail.com (Deepak Nagaraj) — Tue, 11 Apr 2023 00:00:00 +0000

Note: My blog posts are (still) 100% handwritten, without any contribution from an AI model.

Large language models (LLMs) are very powerful, as ChatGPT has demonstrated very successfully. In this post, I will document some caveats, tips and pointers if you are interested in utilizing an LLM for an enterprise use-case.

Tutorial: Understanding Beam with a Local Beam, Flink and Kafka Environment

n.deepak@gmail.com (Deepak Nagaraj) — Thu, 07 Jul 2022 00:00:00 +0000

This is a tutorial-style article. I wrote it in June/July 2022, but found time to clean up and make a blog post only in September 2022. This tutorial is relevant to software engineers and data scientists who work with Apache Beam on top of Apache Flink. Our goal is to set up a local Beam and Flink environment that can run cross-language Beam pipelines. Specifically, in this tutorial, I will discuss how to set up your laptop to run a Beam pipeline that:

Barrett on the Human Brain

n.deepak@gmail.com (Deepak Nagaraj) — Mon, 30 May 2022 00:00:00 +0000

In the past, I have written here about the theory of psychology. One weak point about it is that it’s a lot of, well, theory. However, what if there was an underlying physical reality to it, i.e. to how our brains develop and work? I recently read two books by Lisa Feldman Barrett. I think she provides this much-needed connection.

In Search of Lost Time

n.deepak@gmail.com (Deepak Nagaraj) — Fri, 31 Dec 2021 00:00:00 +0000

As the year 2021 draws to an end, it is a good time to write about In Search of Lost Time. I found a graphic novel adaptation of it and bought it for my fortieth birthday. The title seemed quite apt.

Throwback Thursday: Daylight hours across time and location

n.deepak@gmail.com (Deepak Nagaraj) — Thu, 30 Dec 2021 00:00:00 +0000

Back in 2006, I wrote a script (in Perl!) to plot daylight hours over the year, given a location. From what I remember, I was curious about how it changed with the seasons: was it a straight line or a curve, and how did it look in the polar regions? (If any image below looks blurry, open it in a separate tab or save it to disk to see it in actual size.

Religion as a Salve for Existential Truths

n.deepak@gmail.com (Deepak Nagaraj) — Sat, 25 Dec 2021 00:00:00 +0000

Today is Christmas, an apt time to write a little about religion. I’m not a Christian, I cannot even call myself religious, but more on it in a bit. The Four Existential Truths I have long wondered about the purpose of religion, but a couple of years ago, I found an answer in a book by Irvin Yalom.1 He’s certainly not writing about religion; he’s not a priest or a preacher, but a psychotherapist.

A meditation on time and change

n.deepak@gmail.com (Deepak Nagaraj) — Sun, 28 Nov 2021 00:00:00 +0000

About a year ago, I read this in a book on psychotherapy1: There cannot be change without loss. It struck me as a profound statement, because it carries many implications with it. I want to expand on what change means, looking behind and ahead in time, especially in our times. In retrospective, change is loss We often talk of “big life changes”: graduation, marriage, childbirth, and so on. But it’s wiser to think of them in terms of loss: what did I lose with this change?

Throwback Thursday: Linux in the 2000s

n.deepak@gmail.com (Deepak Nagaraj) — Thu, 04 Nov 2021 00:00:00 +0000

In the years between 2000 and 2005, I spent a lot of time trying out many different operating systems. Linux was a big one: Red Hat, Debian, SuSE. So also the BSD Unices: FreeBSD, OpenBSD, NetBSD. There was Solaris from Sun Microsystems. I also checked out MINIX (educational, from a CS prof), BeOS (good media support) and QNX (real-time OS). I tried dual- and multi-boot installations, also tried installing within VMWare.

Underrepresented Senses in the Electronic Age

n.deepak@gmail.com (Deepak Nagaraj) — Tue, 12 Oct 2021 00:00:00 +0000

There’s a disparity in how our smartphones and computers cater to our senses. They rely heavily on the eyes and the ears. What’s left out is touch, smell and taste. Not strictly a sense, but scale of size is another I would add to this list – it’s the difference between looking at a T. rex in a museum vs a TV documentary. These can be a differentiator for brick-and-mortar stores.

The Father and The Bishop

n.deepak@gmail.com (Deepak Nagaraj) — Sat, 02 Oct 2021 00:00:00 +0000

I recently saw the movie The Father (2020) by Florian Zeller. It’s a poignant movie on the themes of aging and senility. Anthony Hopkins acts as the father who is losing memory, and along with it, a sense of what’s happening around him. To his family and us, it is abundantly clear, but the old man lives in a world of denial. What results is continuously escalating conflict, culminating in a moving climax. In this heartbreaking scene, the old father cries like a baby, and starts wailing “Mother! O Mother!” to the nurse who is caring for him. The nurse, instinctively, comforts him like a child.

Notes from bird-feeding

n.deepak@gmail.com (Deepak Nagaraj) — Fri, 24 Sep 2021 00:00:00 +0000

Homebound in the pandemic, I purchased a bird-feeder last December. I’ve had a lot to observe since then. What follows are notes, photographs, and video clips. I conclude with tips and advice if you’d like to try it as well.

A systems approach to mental well-being

n.deepak@gmail.com (Deepak Nagaraj) — Thu, 02 Sep 2021 00:00:00 +0000

I am an engineer by profession but I like to read about psychology and psychotherapy out of interest. As I have become familiar with it, I’ve found it to be highly logical.

I will specify the system first, and then use it to make some suggestions for our improvement.

I originally started writing this in July 2021 but it took several revisions before I felt it was polished enough to post.

Large hailstorms in the US

n.deepak@gmail.com (Deepak Nagaraj) — Tue, 29 Jun 2021 00:00:00 +0000

Hailstorms can pose a problem, if you’re installing solar panels. Large hailstones, those that are 2 inches wide or larger, can damage solar panels.

If you are solar panel provider, you typically want to play it safe and install where there has been no hailstorms, historically.

Making your program faster by relegating blocking I/O to thread-pool

n.deepak@gmail.com (Deepak Nagaraj) — Thu, 17 Jun 2021 00:00:00 +0000

Sorry about that jargon soup, but this post is technical and is intended for such audiences.

I recently had an opportunity to speed things up by 10X:

Run-time drop from 20 min to 2 min

Selections from Kagga

n.deepak@gmail.com (Deepak Nagaraj) — Thu, 10 Jun 2021 00:00:00 +0000

I posted the verses below originally on the LJ community daily_kagga, almost daily, during 2006-07. I have not altered the content in any way while creating this page. The verses originally are from a philosophical work in Kannada, Mankuthimmana Kagga (1943), by D V Gundappa. 2006/12/17 19:54:00 Reputation ಅನ್ನದಾತುರಕಿಂತ ಚಿನ್ನದಾತುರ ತೀಕ್ಷ್ಣ । ಚಿನ್ನದಾತುರಕಿಂತ ಹೆಣ್ಣುಗಂಡೊಲವು ।। ಮನ್ನಣೆಯ ದಾಹವೀಯೆಲ್ಲಕಂ ತೀಕ್ಷ್ಣತಮ । ತಿನ್ನುವುದದಾತ್ಮವನೆ ಮಂಕುತಿಮ್ಮ ।। There is in Man craving for food, but more so for wealth.

Et Cetera

n.deepak@gmail.com (Deepak Nagaraj) — Tue, 08 Jun 2021 00:00:00 +0000

Although a technologist by profession, I appreciate and enjoy many branches of sciences and humanities. Physics, history, geography, economics, art, literature, philosophy and psychology are all subjects I have often dabbled in out of curiosity. Philosophy and Psychology I think Barrett’s books provide a neurological basis to psychology. Religion may seem obsolete, but I think that it has a lofty purpose: helping us with our existential truths. Some book recommendations while reviewing a book by Lori Gottlieb.

Technology

n.deepak@gmail.com (Deepak Nagaraj) — Tue, 08 Jun 2021 00:00:00 +0000

This section of the site showcases some blog posts and artifacts from software engineering and machine learning. I also throw in some advice for free where it makes sense. Machine Learning A setup to profile NVIDIA GPUs using their software suite (Nsight). I have some implementation notes if you’d like to use large language models (LLMs) at your company. Elasticsearch used to be lexical search only, but ES now allows you to run ingest pipelines and semantic search.

Rules for humans in the age of automation

n.deepak@gmail.com (Deepak Nagaraj) — Fri, 04 Jun 2021 00:00:00 +0000

I recently read Futureproof by Kevin Roose. Kevin is a technology writer for The New York Times. The book has two parts to it; I skimmed through the first part but found the second part interesting. It is a list of nine rules designed to help us, humans, navigate a world increasingly run by machines.

I want to call out four of them that struck me as very insightful.

Detecting irrigation with radar satellite and artificial intelligence

n.deepak@gmail.com (Deepak Nagaraj) — Fri, 07 May 2021 00:00:00 +0000

I’ve previously talked about detecting irrigated croplands in this blog. I used classical machine learning techniques in it, i.e. we decide what features to use, we fit a model or an ensemble of models, we predict. Modern machine learning, or what the press calls “Artificial Intelligence”, has gone way beyond it. In this post, I want to talk about a different approach to detect irrigated croplands. This method gave 95% test-set accuracy on a random sample of 2000 pictures from California.

Gottlieb and other tales of psychotherapy

n.deepak@gmail.com (Deepak Nagaraj) — Thu, 06 May 2021 00:00:00 +0000

Back in 2004, when I was in Bangalore and still fresh out of college, I was mesmerized by a book with a title that asked you to kill the Buddha if you met him. The murderous Zen title notwithstanding, the book was written by a psychotherapist who drew parallels between literary characters and his patients. I wanted to read more such books, but as I said, I was in India and it was 2004. Only in the last year have I delved deep again into the realm of psychotherapy.

Lego piano

n.deepak@gmail.com (Deepak Nagaraj) — Fri, 30 Apr 2021 00:00:00 +0000

I built a mechanical Lego piano. It very much looks like a real piano, both on the outside and inside. It has keys, gears, hammers… its designers are piano teachers. The music comes from a companion Bluetooth app, but the piano makes convincing movements. I’m very impressed. This is my first Lego project, and the fact that I could do something this complex tells you how mature and professional Lego’s assembly instructions pictures are.

Irrigation ML paper published

n.deepak@gmail.com (Deepak Nagaraj) — Sun, 25 Apr 2021 00:00:00 +0000

I’ve previously posted on this blog about a machine-learning model we built to detect irrigation globally.

I’m happy to report that our work is now published as a paper in the journal Advances in Water Resources.

George Saunders: A Swim in a Pond in the Rain

n.deepak@gmail.com (Deepak Nagaraj) — Sat, 17 Apr 2021 00:00:00 +0000

Close to the end of A Swim in a Pond in the Rain, George Saunders looks back on a class from his graduate school. His professor, a short-story writer, gave a reading of Chekhov. Saunders didn’t know much about Chekhov then: he thought Chekhov’s stories were “mild” and “voiceless”. The professor reads three stories, and in doing so, brings Chekhov to life. After that reading, Saunders was not only convinced of the power of the short story, but also “desperate” about writing better short stories himself.

Julian Barnes: The Sense of an Ending

n.deepak@gmail.com (Deepak Nagaraj) — Sat, 10 Apr 2021 00:00:00 +0000

The Sense of an Ending is a slim novel by Julian Barnes. I don’t have much to say about the story, but I was drawn to the main character, Tony Webster.

Ishiguro: Klara and the Sun

n.deepak@gmail.com (Deepak Nagaraj) — Wed, 07 Apr 2021 00:00:00 +0000

We are all familiar with sci-fi stories of evil robotic overlords. I’m not into science fiction as a genre, yet I thoroughly liked the movies Ex Machina (2015) and The Matrix (1999). Both movies are nominally about artificial intelligence, but ask deep philosophical questions.

Ghachar Ghochar, A Novella

n.deepak@gmail.com (Deepak Nagaraj) — Sat, 27 Mar 2021 00:00:00 +0000

I recently read Ghachar Ghochar, a novella by Vivek Shanbhag. The original is in Kannada, but I read the English translation by Srinath Perur.

The novel is slim; it runs to a little more than 100 pages. This is significant, because within these pages, the author still tells a wonderful story. It’s a story that leaves a lot unsaid, for the reader to fill in the details.

About

n.deepak@gmail.com (Deepak Nagaraj) — Sun, 07 Mar 2021 00:00:00 +0000

Hello, I’m Deepak. I work as a machine learning engineer at Citrix. On a typical day, I might be writing models, deploying them into production, enhancing ML infrastructure, or debugging running models. I use Spark, Python, Scala, sklearn, Hive, YARN and Azure for my daily work. I’m passionate about real-time and production-grade ML systems, which demand that you understand both software engineering and machine learning. From 2003 to 2015, I worked in networking companies, where I wrote software for load balancers to make websites load faster.

How we built a global ML model with Google Earth Engine

n.deepak@gmail.com (Deepak Nagaraj) — Wed, 05 Aug 2020 00:00:00 +0000

A couple posts ago, I described the machine learning model that we developed to predict the extent of cropland irrigation worldwide. This post is a collection of all the things I learned about Google Earth Engine that went into running a global-scale model successfully.

The importance of selection effect

n.deepak@gmail.com (Deepak Nagaraj) — Fri, 17 Jul 2020 00:00:00 +0000

It was the first class of a data science course in the MIDS program. Our instructor started off with a question: “Do you think this academic program gives you a better career?” We gave a bunch of answers, quite the ignorant folks that we were. Of course, is this even a question at this point?! Yes, here are all the new things we’re learning. Yes, the instructors are top-notch. Yes, look at the syllabus and the projects, yada yada … nothing surprising there.

The instructor asked, “Yes, but how do you know that the program is making you better? What if you joined the program because you were motivated enough to apply and work through it, and therefore you’d get better in your career anyway?”

Global Irrigation Map, our capstone machine learning project

n.deepak@gmail.com (Deepak Nagaraj) — Sun, 05 Jul 2020 00:00:00 +0000

For our capstone project at UC Berkeley, we worked with the Department of Environmental Studies. We developed a machine learning model that we continued to improve even after the course term, and I’m happy to report that it is now under discussion to be a published paper – a first for the program itself, according to our lecturer.

Global Irrigation Extent, as per our ML Model

Update Aug/5/2020: New blog post: a behind the scenes look on how we built the model.

Lazy evaluation in real life

n.deepak@gmail.com (Deepak Nagaraj) — Sun, 28 Jun 2020 00:00:00 +0000

There are so many great ideas in engineering we can take home and apply to our own lives. Today I will talk about one of them: lazy evaluation.

How are you saving your knowledge?

n.deepak@gmail.com (Deepak Nagaraj) — Sat, 02 May 2020 00:00:00 +0000

We live in a “knowledge economy”. Every day brings with it new information. Do you have an external system to store your knowledge efficiently and effectively?

Notes on Spark Streaming app development

n.deepak@gmail.com (Deepak Nagaraj) — Wed, 18 Dec 2019 00:00:00 +0000

This post contains various notes from the second half of this year. It was a lot of learning trying to get a streaming model working and ready in production. We used Spark Structured Streaming, and wrote the code in Scala. Our model was stateful. Our source and sink were both Kafka.

Privacy in today's age with a SOCKS proxy

n.deepak@gmail.com (Deepak Nagaraj) — Wed, 07 Aug 2019 00:00:00 +0000

Say you are at a cafe, and you want to surf the Web. But the WiFi is not secure. Or say your company lets you bring your laptop, but what if its firewall has blocked your favorite website? Is there no hope, besides paying $15 to a VPN provider?

There is, and it costs about $3.50 per month as of this writing.

The Spark tunable that gave me 8X speedup

n.deepak@gmail.com (Deepak Nagaraj) — Wed, 07 Aug 2019 00:00:00 +0000

There are many configuration tunables in Spark. However, if you have time for only one, set this one. It made a streaming application we run process data 8X faster. That’s 800% improvement, no code change needed!

Getting top-N elements in Spark

n.deepak@gmail.com (Deepak Nagaraj) — Sat, 11 May 2019 00:00:00 +0000

The documentation for pyspark top() function has this warning:

This method should only be used if the resulting array is expected to be small, as all the data is loaded into the driver’s memory.

This piqued my interest: why would you need to bring all the data to the driver, if all you need is a few top elements?

The answer is: it does not load all the data into the driver’s memory.

Livy is out of memory

n.deepak@gmail.com (Deepak Nagaraj) — Sun, 11 Mar 2018 00:00:00 +0000

Spark jobs were failing. All of them. The data pipeline had stopped. This is a tale of high-pressure debugging.

Accessing home computer from anywhere

n.deepak@gmail.com (Deepak Nagaraj) — Sun, 26 Nov 2017 00:00:00 +0000

Do you sometimes want to access your home computer from an outside network? Maybe you use another system, but you do not trust it and would prefer your home computer for some workflows?

This post outlines the steps to make such access possible.

The program that would not go away

n.deepak@gmail.com (Deepak Nagaraj) — Fri, 17 Feb 2017 00:00:00 +0000

This post is about a program hang. The hang was in the Python process that was running Ansible scripts. The problem was hard to debug and had me go back to Unix textbook.

Correct way to create a directory in Python

n.deepak@gmail.com (Deepak Nagaraj) — Thu, 02 Feb 2017 00:00:00 +0000

Can you see the problem with this code? It comes from Ansible, v2.1.1.0.

if not os.path.exists(value):
     os.makedirs(value, 0o700)

It’s quite straightforward. It checks if a directory path exists. If it does not, then it creates the directory path, similar to mkdir -p. What could be wrong?

Getting rid of unused virtual disks on XenServer

n.deepak@gmail.com (Deepak Nagaraj) — Wed, 01 Feb 2017 00:00:00 +0000

A continuous test server I’d set up had stopped working. The XenServer on which it was running had a 1TB disk: and it was full. What’s going on?

Log rotation, no code change needed

n.deepak@gmail.com (Deepak Nagaraj) — Tue, 24 Jan 2017 00:00:00 +0000

This post shows you how to rotate old logs from your application. There is no change to application code. There is no specialized logging library or framework needed. It works for any language, on standard Unix platform.

Resetting a TCP connection and SO_LINGER

n.deepak@gmail.com (Deepak Nagaraj) — Fri, 21 Oct 2016 00:00:00 +0000

Can you quickly close a TCP connection in your program by sending a reset (“RST”) packet?

Program crashes, only in 64-bit mode

n.deepak@gmail.com (Deepak Nagaraj) — Fri, 14 Oct 2016 00:00:00 +0000

This is a story of a program that worked, until it broke on a 64-bit platform.

ps | grep is broken on FreeBSD

n.deepak@gmail.com (Deepak Nagaraj) — Mon, 10 Oct 2016 00:00:00 +0000

It is true. Even on the latest FreeBSD 11.0 (I checked the source tree).

Debugging emacs slowness

n.deepak@gmail.com (Deepak Nagaraj) — Thu, 29 Sep 2016 00:00:00 +0000

I use Emacs (emerge) while merging files. Today, when trying to merge some Python code, I found that it was taking exceedingly long time. It was blank for 5 minutes and counting.

Weird PID files

n.deepak@gmail.com (Deepak Nagaraj) — Thu, 29 Sep 2016 00:00:00 +0000

This is a story of multiple processes running on a system, but with empty PID files.

It took a lot of debugging.

Don't hang up on me

n.deepak@gmail.com (Deepak Nagaraj) — Sat, 30 Jul 2016 00:00:00 +0000

Can you tell me when a shell sends a “HANGUP” signal to a process? What happens if there is a pipeline? What if you prefix this pipeline with the “nohup” command?

Rotating the log makes it empty

n.deepak@gmail.com (Deepak Nagaraj) — Fri, 29 Jul 2016 00:00:00 +0000

If you don’t do log rotation right, you may have a full hard drive and a ghost file.

Socket timeout

n.deepak@gmail.com (Deepak Nagaraj) — Wed, 29 Jun 2016 00:00:00 +0000

Have you run into a problem that happens only once in a while, and you never seem to have enough information to figure it out? We all have. This is one such tale: except that it has a happier ending.

Build fails, only for you

n.deepak@gmail.com (Deepak Nagaraj) — Thu, 16 Jun 2016 00:00:00 +0000

As programmers, we face it many times: a build fails on only our system. This is one such tale.

You got too many open files

n.deepak@gmail.com (Deepak Nagaraj) — Sat, 21 May 2016 00:00:00 +0000

This is a tale about a hopelessly bad error message.