Vamsee Jasti2021-04-25T21:34:47+00:00jasti.github.ioVamsee JastiPaternity Privilege2019-09-02T00:00:00+00:00jasti.github.io/posts/paternity<center>
<img src="/images/blog/paternity/sleep_log.png" width="800" height="800" />
<br />
<i> A different kind of streak</i>
</center>
<p><br /></p>
<p>Before Miles, our son, was born I didn’t have a strong opinion about wanting to have kids. Not that I was against it but I didn’t see a strong reason to voluntarily give up a life I dearly enjoyed for an unknown quantity. This feels preposterous in retrospect. But so do most things that shoulder the weight of their consequence.</p>
<p>The last two months have been a mind-bending experience. For once, I was in the 1%. The 1% of fathers that get to be the primary caregiver for more than 2 months with their new born. I wish everyone has this opportunity because it was nothing like I had imagined it when I started.</p>
<p>I was excited. I was going to spend the next two months with the person that evolution had programmed in me to love more than myself. This was also the longest I was going to be away from work during my adult life. It was also the beginning of my first summer in the East Bay in the Bay Area where there is an actual summer, unlike San Francisco.</p>
<p>I’ve run races that lasted more than 18 hours with a 40 pound rucksack on my back. Undoubtedly, being a primary care giver to a 4 month old has been the most mentally and physically challenging thing I’ve ever done.</p>
<center>
<img src="/images/blog/paternity/bricks.jpg" width="600" height="600" />
<br />
<i> 40 pounds of tapped bricks for the run</i>
</center>
<p><br /></p>
<p>Having said that, I’m grateful for the experience because it has helped me:</p>
<ul>
<li>
<p>get a sense of what my partner went through during her maternity leave. The loneliness, boredom, monotony and the lack of any positive feedback from the baby.</p>
</li>
<li>
<p>imagine what it must be like for single parents. Like being on a treadmill that you can’t step off of.</p>
</li>
<li>
<p>empathize that money doesn’t solve all problems but it solves many parenting problems. It was the best crystallization of how poverty begets poverty. It’s difficult to work hard & take professional risks when stressed about being able to afford buying diapers or formula.</p>
</li>
<li>
<p>realize that days go by in a blink while giving care to newborns. I had plans to write everyday, code a hobby project, work on gymnastics skills and be ultra productive while Miles napped. But I didn’t do any of those things. All I wanted to do was sit with a cup of coffee enjoy doing nothing, recovering from the energy I had to bring entertaining the baby when he was up. Or at best prep the house/ cook healthy meals for our family during his naps.</p>
</li>
<li>
<p>understand that it’s possible to seek joy without being Productive or creating something tangible. By the end of it, the only thing I had to show for was a healthy, (mostly) happy child that was 2 months older. It took a few weeks to be okay with (and even proud of) this.</p>
</li>
<li>
<p>internalize that people are all wired different. Some may enjoy taking care of a baby any given day over, say spend a day hiking in Yosemite. The former wasn’t me but forcing myself to be a different person has made me a different person. It was the best of reminders (as I approach peak filter-bubble age) to challenge myself to new experiences and be open to different perspectives because the most fulfilling states of being can be outside of one’s default reality tunnel.</p>
</li>
</ul>
<p>I think raising a newborn is a lot of joy and little fun, bubble wrapped in a constant state of anxiety.</p>
<p>Every decision can be like answering a multiple choice test that you’ll never know the answers to. But maybe that’s actually what makes the whole thing rewarding. There are no right answers because everybody gets different questions and the answers are subjective. There are no rules to this thing and most parents jump off the cliff and seem to figure out the parachute on the way down. “Mistakes” will happen. Some more consequential than others. I’ll never forget how bad I felt when I let him bang his head on the side of a table. There was no way I could have prevented it but it’s still hard to let go of what happened.</p>
<p>For my friends who don’t have kids, there have been a few times when I sounded like a lunatic describing the joy in raising a child. I don’t sound far off from someone on a psychedelic trip talking to a sober crowd. On a trip, you could look at the most ordinary of things, like leaves dancing to the wind and realize the weight of all things that had to fall in place in the multiverse for the observer to witness that miracle. Most people, including me, brush off such experiences 100s of times without acknowledgement. Can you call it a miracle if it’s commonplace? <br /></p>
<p>I feel like raising a child puts parents into a 24/7 psychedelic state of awe and wonder, making you acknowledge the daily miracles in the most banal of beings - like when your baby smiles at you.</p>
<center>
<img src="/images/blog/paternity/miles_6mo.JPG" width="400" height="600" />
<br />
</center>
<p><br /></p>
<p>That’s it for now from me as I enthusiastically tie my laces to get back to work. I’m grateful for being able to bond with him so deeply. I’ll miss him but I know I’ll cherish the precious hours I’ll get to spend with him for the next 17.5 years building on top of our relationship.
<br /></p>
<p>I hope you’ll tell me more about what I don’t know about parenting.</p>
Maui & the Big Island2018-05-21T00:00:00+00:00jasti.github.io/posts/hawaii<p>We just spent a week in paradise and thought I’d note someplaces we enjoyed the most. If we didn’t absolutely love something about a place, it’s not on the list. Obviously these recommendations are heavily skewed towards what my wife and I like to do which mostly revolves around being outside as much as possible, eating good food and drinking good coffee & beer. We are both vegetarian and although Hawaii’s cuisine is fairly meat-centric, we were able to find some pretty amazing vegetarian options. Also, normally we’d hike a lot more, but I sprained my left ankle pretty bad on the first day on a hike and chanced my right foot with a sea urchin while surfing. There are probably many amazing hikes I’m not covering here.</p>
<h2 id="maui">Maui</h2>
<p>Maui is the most well-rounded of all 4 islands and this place seemed to have something to do for everyone.</p>
<h3 id="things-to-do">Things to do</h3>
<ul>
<li><strong>Iao Valley Hike</strong>: It’s a fairly short hike while being surrounded by magically large mountains and waterfalls everywhere you look. You also get to read about a bunch of history as you walk along the hike which makes it special. It reminded me a lot of Machu Picchu and dare I say it looked almost as beautiful as Machu Picchu without needing to hike for days.</li>
<li><strong>Upcountry</strong>: That’s what they call the region that is on top of the hills. A few things to do there include:
<ul>
<li><strong>Lavender farm tour</strong>: Lavender nerds would lose their shit here. There are 7 type of lavender, apparently. They make all sorts of things with lavender. Perfume, butter, chapstick, tea & coffee. But the best of all was the lavender scones that you get to enjoy with passion fruit jam. You get to eat these while getting a 270 degree view of all of Maui, while being surrounded by a well-manicured garden.</li>
<li><strong>Maui wines</strong>: If you are into tropical wines, you will love this. They have a neat wine tasting option. But the best part about it is the drive up there and greenery. It felt like some remote part of Scotland.</li>
<li><strong>Surfing Goat cheese tour</strong>: Ton of baby goats in a place with surfboard motif. Need I say more? Also offer tours and tastings.</li>
</ul>
</li>
<li><strong>Beach Walks</strong>: We enjoyed taking long walks along the beach at sunset everyday. You just can’t go wrong with this but, the two places we particularly liked were Wailea and Kaanapali beach because they have nice paved paths that seemed to never end.</li>
<li><strong>Snorkelling</strong>: My only other point of reference for snorkeling is from the Cenotes in Tulum. Although Tulum wins for the location, Maui wins for the variety of fish you get to see. Some colors are hard to describe. Some were so bright and made up of many colors that they put incredibly beautiful flowers to shame. Bonus, at Napili Bay, we also swam alongside a turtle that was about 4 feet for about 20 mins with nobody else next us.</li>
<li><strong>Drive to Hana</strong>: This was a lot of driving in one day, but it was worth it. Waterfalls everywhere you look. Hana, the town itself isn’t much to see and it would be all worthwhile if you aren’t in a rush and are able to stop a number of times along the way and absorb your surroundings. Below are a few highlights:
<ul>
<li><strong>Twin falls</strong>: A mini park with almost too many waterfalls. Also has a hike which we didn’t end up doing but heard good recommendations about. Recommend diving into the waterfall. I live to tell the tale.</li>
<li><strong>Wai’anapanapa State Park</strong>: Cool black sand beach in a remote state park.</li>
<li><strong>Oheo Gulch</strong>: 7 “layers” of waterfalls that you could also swim in.</li>
<li><strong>Haleakala State Park</strong>: Super gigantic state park which we didn’t cover much but the places we saw were incredible.</li>
</ul>
</li>
<li><strong>Banyan Tree in Lahaina</strong>: All of Lahaina is pretty touristy but this Banyan tree that spans 100s of meters is worth checking out.</li>
</ul>
<h3 id="food">Food</h3>
<ul>
<li><strong>Maui Brewing co tour, food and drinks</strong>: I was pleasantly surprised with how many business care about sustainability in Hawaii. We learnt quite a bunch about this brewery and fell in love with the company ethos. They do some amazing tropical beer flavors like Coconut, Mango and Passion Fruit and of course, the usual lagers and IPAs but with their own unique twists. We went back 4 times during our 4 day trip.
-<strong>a’a roots açaí bowls</strong>: Acai meets tropical fruit presented with flowers.</li>
<li><strong>Maui-grown Coffee</strong>: Family owned coffee shop. Try the peaberry coffee which is a type of coffee made from beans when there is only a single bean in the cherry. I tried to not get used to this expensive taste.</li>
<li><strong>Paia Bay Coffee</strong>: Mostly for the ambience but the coffee was solid.</li>
<li><strong>Coconut Glenn’s</strong>: It’s on the drive to Hana at mile 27. You’ll find a tiny hut that sells delicious vegan coconut ice creams, coconut water etc.</li>
<li><strong>Eat ton of local fruit</strong> : This applies for both islands. The quality is simply unmatched. Especially Pineapple, Papaya, Mangoes and Lychees. The best place to buy them are at random roadside stands when driving. You’ll find some great deals and quality.</li>
</ul>
<p>If you decide to go to more than one island. Use Mokulele airlines. They only fly 10 seater commuter airplanes which means that you have hassle free boarding and arrival process and you get the best view of the islands during take-off and landing.</p>
<h2 id="big-island">Big Island</h2>
<p>It was like nothing else I had seen in my life and certainly not what I expected from an island in Hawaii. The Kona airport is surrounded with black lava with almost no vegetation or fauna. I later found that only a part of the island looked like that. There were parts of the island that were literally getting formed in front of our eyes. It’s hard not to contemplate our origins, evolution, our place in the universe and many un-mundane questions while exploring the big island at every turn in the road.</p>
<h3 id="things-to-do-1">Things to do</h3>
<ul>
<li><strong>Mauna Kea Observatory for Star Gazing</strong>: Absolutely mind blowing. You can drive to this observatory and local volunteers have their telescopes scoped at planets and the moon for you to checkout. At this height, 9000 ft, the stars don’t twinkle much because of lower distortion in the atmosphere. And since there is no light pollution, you are able to see millions of crisp looking stars. Bucket list item.</li>
<li><strong>Snorkeling with Manta Rays</strong>: Another bucket list item. I’ll maybe write a whole post about it. Just enough to say that we spent about 45 mins in the middle of the ocean during sunset with Manta rays circling us (including one touching me). Uterly meditative experience.</li>
<li><strong>Kahalu’u bay for surfing and snorkeling</strong>: Nice beginner break with colorful fish right beneath you. The water is green in color, so you can snorkel and look at the first while waiting for the waves. Only downside is the bay is filled with sea urchins. Wear thick booties.</li>
<li><strong>Pololu valley</strong>: Magical remote valley that ends in a beach with dramatic cliffs with almost no-one there. Couldn’t find a more perfect place to just hang-out, read and relax.</li>
<li><strong>GreenWell Coffee farm tour</strong>: One of the first coffee roasters to produce 100% Kona coffee. The tour is free and the farm has a bunch of other fruit trees too (yes, I like fruit).They have coffee tastings and a tiny shop selling coffee and fruit.</li>
<li><strong>Drive on Kohola Highway</strong>: If you’ve visited Pololu valley, drive on this highway on the way back. You’ll see ridiculously large mountains appearing and disappearing with the landscape around you changing about 5 times from ultra dry to the lushest green.</li>
<li><strong>Makalewana Beach</strong>: I generally don’t like to just “hang out” on the beach. But that is exactly what we did here. Hands-down the most beautiful beach I had ever been too. The bluest of blue waters with fine white sand. The beach itself is only about a few hundred meters long and you need to hike 2.5 miles on foot to get here which kept the crowds away. That makes it extra special.</li>
</ul>
<h3 id="food-1">Food</h3>
<ul>
<li><strong>Big Island Brew House</strong>: Had some really interesting brews and solid food - a mix of mediterranean-mexican. They seemed to have won a ton of awards for their beer.</li>
<li><strong>Under the Bodhi Tree</strong>: Best vegetarian food in the 4 islands. Excellently prepared with organic ingredients. It’s a bit of drive from Kona but worth it.</li>
<li><strong>Kona Brewing Company</strong>: This took place of Maui brewing Co during our stay in the big island. Longboard lager is my favorite lager, so having this on tap at the mothership was a treat. Also found a new favorite - the Koko Brown with a ton of coconut and Kona coffee.</li>
<li><strong>Fresh off the grid</strong>: Go here on your drive back from Polollu valley. A tiny smoothie and fresh fruit shop on an open farm facing the ocean. The smoothie was incredibly fresh.</li>
<li><strong>Basik cafe for Acai Bowls</strong>: Solid Acai bowls in downtown Kona.</li>
<li><strong>Daylight Mind Coffee and the views</strong>: The view from this cafe is unbeatable. Right on the ocean with some wonderful 100% Kona drip coffee.</li>
<li><strong>Kaya’s for coffee and breakfast</strong>: Organic breakfast cafe with excellent coffee. They also had bulletproof coffee with Ghee which was quite good. It’s a recipe I’m bringing back home.</li>
<li><strong>Gertrude’s Jazz Bar</strong>: Live jazz overlooking the ocean. Quite a change in the jazz bar scene compared to the underground jazz bar scene in New York City. Good cocktails too. We went around sunset and stayed through twilight. Perfect way to spend an evening.</li>
</ul>
<p>Reach out if you think other places should be on this list. I don’t think this will be the last time we’ll visit these two islands.
Wish you safe travels and good memories!</p>
Attachment to Staying Present2018-04-09T00:00:00+00:00jasti.github.io/posts/present<p>For those of you who have explored Buddism, how many times have you heard that Buddhism is about being in the present moment?
What often doesn’t get talked about is Buddhism’s relative priorities. Being present is an important concept of Buddhism but it’s not more important than the notion of clinging. Described differently, attachment.</p>
<p>We all tend to cling to more things than we realize in our day to day. For example, I’m currently clinging to writing this article. In the same way, society has reinforced that being present is the ‘it’ in Buddhism and therefore everyone should let go of responsibilities, past and the future and stay focussed on the present. Staying present brings a lot of clarity to one’s actions. It makes one experience the real you instead of passing life in a literal dream state.</p>
<p>If you are practical though, our day to day living requires planning and learning from our mistakes. A better way to approach being present is doing so without being attached to it. It’s the ability to move between different states of present, past and future to do what is appropriate without clinging.</p>
<p>Ironically, in order to not cling, one needs to be constantly aware of not clinging which is in itself a form of being present but just much more lightly than one would be while being capital P, Present.</p>
<p><br /></p>
<center><img src="/images/blog/present/present.png" width="400" height="400" /></center>
<p><br /></p>
Stories We Tell Ourselves2018-03-11T00:00:00+00:00jasti.github.io/posts/stories<p>If you haven’t read the book Sapiens, I highly recommend you do. It’s one of my top 3 all time favorite books. I took away one concept from Sapiens that I think about almost every day. It’s about the human ability to take part in a story.</p>
<p>Believing in stories we tell each other has let homo sapiens conquer the universe. No other species has the ability to either tell or listen to stories. So what’s unique about participating in stories?</p>
<p>Stories have led to humans to cooperate and build something bigger than the species. The US constitution is a story, democracy is a story. The New York Times tells stories and how people talk about the times is a story. What North Korea tells it’s people is a story. Marriage is a story and so is product development.</p>
<p>Biases are built based on stories people listen to, the people they interact with and the opportunities they are given. The better the story, the more number of listeners and a better grip on those listeners.</p>
<p>One neat way to decide if you should believe a story is to go down to the first principles and find out if what’s being said is a fact vs an opinion. Problem is, not all stories can be traced down facts, especially those with an indeterministic outcome.</p>
<p>Sometimes, the facts are there but you’d likely only spend the time to get down to the facts if your life depended on it. Humans by nature like certainlity and feel comfort in arriving at a judgement which makes it hard to detach from the narrator, especially with your pre-conceived biases. The stronger the attachement to the story the more you don’t want to listen to the alternative narrative.</p>
<p>Reading Sapiens was like taking the red pill from Morpheus. You won’t be able to help but ask yourself whenever you engage in a new conversation if the story is really real or if there is an alternative version to the story.</p>
<hr />
If not this, what else?2018-02-17T00:00:00+00:00jasti.github.io/posts/moment<p>Every moment is unique and nobody is exempt from the unfolding of each moment. The fleeting feeling of the moment can feel isolating but the impermanence of the moment is what makes it precious. A lot of pleasure can be had by being there when one moment freshly blooms after another.</p>
<p>The immediate question I have is - so what? What’s the point of welcoming the next moment? To that, it’s worth asking, if not this, what else? If I couldn’t stay here and watch the events unfold, does it even matter that it happened?</p>
<blockquote>
Only that day dawns to which we are awake. - <b> Henry David Thoreau, Walden </b>
</blockquote>
<p><br />
If not this, what else?</p>
<center>
<img src="/images/blog/moment/amsterdam.jpg" width="800" height="800" />
<br />
<i> A cold February morning in Amsterdam </i>
</center>
<p><br /></p>
Running a 50 mile ultramarathon2017-11-24T00:00:00+00:00jasti.github.io/posts/ultra<p>Four months ago, I signed up to run the The North Face 50 Mile Endurance Race in Marin County. I signed up the very next day after I went trail running for the first time. The Marin Headlands are so beautiful that I wanted a challenging excuse to spend more time exploring them. At 10,500 elevation delta, I hear that The North Face 50 is one of the most challenging 50 mile races in the US.
<br /></p>
<center><img src="/images/blog/ultra/bib.jpg" width="400" height="400" /></center>
<p><br />
I had run a few marathons in the past but didn’t think running almost twice the distance was even possible. However, the ultramarathon somehow made it on my bucket list. A 50 mile race is more challenging than a marathon in two ways. First, obviously, it’s almost double the distance of a marathon and second, the hills. Most ultramarathons are on trails, and most trails have hills. Muscle fatigue occurs much faster on hills compared to road.
<br /></p>
<center><img src="/images/blog/ultra/marin2.jpg" width="400" height="400" /></center>
<p><br />
My training journey deserves an entire separate post. I’ll just say that it was extremely rewarding because I saw progress every single week. Onto the race.</p>
<p>The night before the race, I set my alarm for 2 AM. What most people don’t realize is how early one needs to wake up for a 5 AM start. I went to bed at 8:30 PM but couldn’t fall asleep until 11. Must have been the nerves. I woke up at 1:50 AM, even before the alarm. It’s funny how your unconcious body knows this entirely man-made concept called time. I had a protein shake, picked up a car rental, picked up my friend Lorenzo who was also running this race and arrived at the start line at 4 AM, an hour before the race start time.</p>
<p>It was cold - in the 40s. Nearly 500 runners were all huddled around 6 heat lamps in the vast open grounds. The heat lamp to runner ratio was abysmal. To make matters worse, I realized I dropped my sunglasses somewhere. The thought of staying out in the sun for 12+ hours without eye-protection was not appealing. Not the way I wanted to start this race but what choice did I have?</p>
<p>The hour went by fast chatting with Lorenzo, our training buddy Ken and other people we didn’t know who were closely huddled in with us. The trail running community is supportive and laid back. Unlike road races, you aren’t out there to compete with other people - just with yourself. Another reason why I enjoyed this sport so much.</p>
<p>When I got to the start line, about 10 seconds before my wave started I realized I forgot to stretch because of all the excitement. My biggest worry about not finishing the race was about my IT Band tightening up - which it had in both 32 miler training runs. Shit, strike 1?</p>
<p>The race started off with a huge climb. I walked it, even though every ounce of my body wanted me to go charging to conquer the hills. I reminded myself that the race didn’t truly begin until mile 25 - that was where the uncharted territory began.</p>
<p>Running in the dark was special. It felt like a dream. I wasn’t sleepy or sleeping, just in an alternate dimension of awake. It was a clear night so the stars were bright and there were many.</p>
<p>The next 10ish miles were in auto-pilot. Hit some monster hills but disciplined myself to power hike all the uphills - which roughly ends up being the same pace for an average runner when going uphill. Skipped the first 3 aid stations but kept hydrated & chomped away at my nutrition bar stash. Of things one can multi-task at, eating while running downhill must be one of the hardest.</p>
<p>More auto-pilot until mile 20. Striked up a conversation here and there with other runners. Miles tend to fly by effortlessly when you engage in a deep conversation while running - especially if it’s about running. Talked to this 50 year old woman, Andrea, who was running her 4th 50 miler in the last 4 weeks. I wanted every misogynist I’ve met in my life to meet her.<br />
<br /></p>
<center><img src="/images/blog/ultra/marin.jpg" width="400" height="400" /></center>
<p><br />
Mile 21, hit a monster wall. Felt like someone strapped on two 10 pound bracelets to my ankles. Struggled through it. Slow, but didn’t stop. Since my legs felt so heavy, I wasn’t lifting my feet off the ground.</p>
<p>Out of nowhere, my left leg caught a tree root which sent me leaping forward, landing on my right knee. Strike 2? My left ankle straightened to an 180 degree angle along with sharp pain. Was sure I broke my ankle. A thought flashed wondering how I’d even get to the aid station for medical help. Came up on all fours and began rotating the ankle slowly. I didn’t see my ankle swell and the pain began to subside. Began slowly walking and then tried running. Phew. My ankle started to feel fine, but mentally hit a new low. Pep talked myself. I wasn’t a self-talker but training for this race has made me realize how important it is to consciously engage with oneself in moments like these when everything in you wants to give up. Training for an ultra is much more mental than it is physical. If you do it right, it teaches you be an irrational optimist. I treated myself to some music from the iPod to help push through the slump.</p>
<p>The next 4 miles were the toughest miles of the entire race. It was probably due to a combination of a steady incline, some cramps and haphazard tree roots on the trial.</p>
<p>Somewhere around mile 25, it felt like I passed through something. I got my stride back. I thought of Jon, my friend, who was going to pace me from mile 27 to 41. The trail turned from an uphill to a slight decline for as far as I could see. No one around me. Just me, the trail and the Pacific Ocean ahead of me. Unexpectedly, tears rolled down my face. I hadn’t experienced anything like it in my life. Something inside me told me how proud it was of me. I tend to be so hard on myself when training, especially during the past 4 months that, I think, this was a way for something in me to recognize and feel grateful for all my body had humbly put up with. I realized what self-compassion felt like in its purest sense.</p>
<p>I met Jon at the Stinson Beach aid station within a few miles. The next few miles were a tough part of the course because we hit all stairs gaining about 1500 ft of elevation without a break. It didn’t matter because Jon did an incredible job taking my mind off the race and kept my mind off the climb. My hamstrings cramped a few times. Jon came prepared with salt tabs which probably saved my race. It’s funny how such a minor detail can demolish 4 months of hard work and your race. More ridiculous elevation gain and loss but we took it all with our chins up. You make the best sort of friendships when you go through such a thing together.
<br /></p>
<center><img src="/images/blog/ultra/jon.JPG" width="400" height="400" /></center>
<p><br />
Mile 41, Jon dropped off at the Tennessee Valley aid station as planned and Kristina joined me to pace me to the finish line. I was delighted to see her. I find a second (or third? fourth?) wind in me. Power hiking all the uphills but pushing 7 min miles on the downhill. We reached the most beautiful view of the Golden Gate I had ever seen. It was so good we couldn’t miss a selfie op.
<br /></p>
<center><img src="/images/blog/ultra/kristina.JPG" width="400" height="400" /></center>
<p><br />
The last part of the race made you run over the Golden Gate bridge. Made a final push to beat about 3-5 more runners in the last 2 miles.
<br /></p>
<center><img src="/images/blog/ultra/goldengate.JPG" width="400" height="400" /></center>
<p><br />
There I was. The finish line, I made it. Accomplished the goal that seemed impossible just 4 months ago. However, the first thing I thought of as I crossed the finish line was how conservative I had been the entire race. What an anticlimax. Did I sandbag it? I had more juice left. Should I have gone faster? It felt like I could have gone at least another 20 to 30 miles but who knows - at these distances, anything can happen. I reminded myself that it didn’t matter if I had finished earlier or run faster. I didn’t sign up for that game. I signed up to finish running 50 miles without getting injured.</p>
<p>Given that my goal when I signed up for the Ultra was to “just finish under the 14 hr time cap,” a time of 10 hrs 31 min was a phenomenal success.</p>
<p>My body type isn’t ideal for running. I’m tall, weighing close to 200#s, so the odds were against me doing well in this race. My biggest accomplishment though wasn’t about finishing this race at all. It was about finding the joy in running, getting to know myself better, exploring the trails and making incredible friends. Although I’ve checked this bucket list item, I’m not hanging up my running shoes yet.
<br /></p>
<center><img src="/images/blog/ultra/finishline.jpg" width="400" height="400" /></center>
<p><br />
Was it all worth it? Heck YES.</p>
<hr />
Why treating animals well today may save you tomorrow2017-11-07T00:00:00+00:00jasti.github.io/posts/alpha<p>Have you heard of <a href="https://medium.com/intuitionmachine/the-strange-loop-in-alphago-zeros-self-play-6e3274fcdd9f">Alpha Go Zero</a> (AGZ) yet? It’s hands down better than the regular Alpha Go which beat the best Alpha Go human player a year ago. The most fascinating thing about the achievement is its ability to self-learn based on logic. AGZ is at a degree where a human is unable to reverse-engineer why it made a move.</p>
<p>It’s inevitable that machines will become smarter than human beings. We can argue about the when but given breakthroughs like this, it’s within decades, not centuries. One of two outcomes is likely:</p>
<ul>
<li>machines end up self-learning to a degree that they find a way to control humanity</li>
<li>an altogether new species of human beings fuses with self-learning machine with tech like <a href="https://waitbutwhy.com/2017/04/neuralink.html">Neuralink</a>.</li>
</ul>
<p>In either case, let’s name the outcome of this evolution, Homo Machines. Homo Machines will be able to compound learn so they will be quite different from humans in just a few years.</p>
<p>So here is my main argument, we humans decide to treat a large majority of the animals so poorly, primarily via factory farming and inhumane hunting, even though we have plenty of DNA in common. This includes commonalities like the ability to feel emotions, pain, hunger, love and jealousy. By this logic, the most likely scenario is for Homo Machines to also treat humans the same way as we do, say, poultry in factory farms.</p>
<p>Some might argue that poultry isn’t close to the top of the food chain or have nearly as much intelligence, so they deserve it. Unfortunately, humans could be in the exact scenario as poultry in a few decades because of the machine’s ability to become intelligent recursively.</p>
<p>What if we changed our way today and we begin treating our animals like we want to be treated ourselves? If yes, is it possible that our training data we are feeding to these machines over a long enough period of time would set a new precedent to treat those lower in the food chain more thoughtfully and as a result, in a few decades, although subservient, we’ll co-exist with Homo Machines?</p>
<p>Given how Homo Machines are fundamentally based on rationale and logic, it’s a bet worth a gander.</p>
<hr />
Thirty2015-07-12T00:00:00+00:00jasti.github.io/posts/thirty<blockquote>
Life is essentially a cheat and its conditions are those of defeat [and] the redeeming things are not “happiness and pleasure” but the deeper satisfactions that come out of struggle. - Scott F Fitzgerald
</blockquote>
<p><br /></p>
<p>I turned thirty last week. Thought it would be interesting to capture my current sense of thinking about the various areas in my life. A la “state of mind.”</p>
<p>Thirty is not old, but it’s definitely not young. I’ve heard people say that their 20’s is the best decade of their life, but I don’t agree. I really enjoyed my 20’s, but knowing what I know now, I’d rather be here now.</p>
<p>For example, it’s silly, but one thing I used to worry when I was 23 was that I wouldn’t (or shouldn’t) go to clubs in my 30s. I worried that I won’t be able to enjoy my <a href="https://open.spotify.com/user/123544547/playlist/1JfP7FUMUcXmhXbSkCvwR3">music</a> as I did back then. Fast forward 7 years, and THANK GOD I don’t like to go to clubs anymore. It’s probably one of the worst experiences I could ever have while paying a lot to spend time in a crowded room. I still enjoy the music and it’s been replaced with loud music on good speakers in my living room while cooking. Sometimes while sipping a beer. Sometimes alone.</p>
<p>Maybe that’s how most age related worries turn out to be. Illusions. Things like physical appearance, physical agility, money, sanity and a sense of belonging. Or maybe I’m in denial telling myself everything is going to be okay. I don’t know, but what choice do I have anyway?</p>
<h2>Work</h2>
<p>In my first week at Google, I read an internal post by someone describing Google as a ‘deeply magical place.’ He made a very logical reason for why he thought so, but I couldn’t help but think that this guy was addicted to the Kool-Aid. I’m 6 months into the job, and I now too think that Google is a deeply magical place. This deserves a post on it’s own, so I won’t go into the details here but I’ll say this much - almost all work related problems I have right now are all ‘first world problems.’ If I ever genuinely complain, I give full liberty to the Humility Fairy to smack me on the head.</p>
<p>I think I have a fairly good idea about what is important for me at work. No, it’s not about learning the next cool programming language. It’s more modest than that. Find interesting things to do. Have the courage to stretch myself as much as possible, but not take myself too seriously in the process. Easier said than done.</p>
<h2>Body</h2>
<p>I feel pretty good about my body. I still give 100% to every workout, every day. But I notice my body is taking longer to recover. I can’t think of doing two workouts a day during the weekends anymore. Maybe it is my body, but maybe it’s just that my head has not been in the right place. Since we moved so many places over the last year, I haven’t been able to commit to a goal to focus on. I’d like to think that it was a blessing in disguise cause I have discovered track sprint training, Rocket Yoga and core strengthening body weight stuff which I wouldn’t have focused otherwise. Staying physically fit was much easier when having a routine.</p>
<p>My mindfulness practice has been a total game changer in the last year. A lot less autopilot and therefore a lot more living with purpose. Meditation has calmed the mind and brought clarity amongst the brain chatter. It has allowed to me to visualize what I need, reason why and silently go get it. I am quite far away from any sought of sustained stability but I like where this is going.</p>
<h2>Relationships</h2>
<p>My roller coaster cart is starting to slow down and I finally feel like I can make some time to write again. Kristina and I left New York in October of last year thinking we would never live in New York again. We had endured enough of the miserable winters. We picked Austin and off we went. We sold everything we owned, except for 5 boxes and 2 suitcases and stood at my brother’s doorstep in Texas. We moved back to New York after exactly 4 months of living in Austin. Not because we didn’t like Austin. In fact, we’ve scoped Austin as a place we might eventually end up in. I got offered a job at Google and I couldn’t say no. Thankfully, my lovely supportive wife agreed and here we are. We ended up staying at 5 different places during the round-trip including my brother’s for a couple of weeks and a temporary place for 2 months after we were back in New York City. By far, the biggest thing we missed about New York were our friends. It’s amazing how much your quality of life can improve when you know you are around people who have your back. They make the roller coaster of change doable, and even fun.</p>
<p>Nothing makes me more happy than being beside Special K. It’s the one thing I look forward to everyday coming back home. I try and avoid spending time with people I don’t like and be around people I do like. Other than enjoying a workout the most fun thing to do these days is have a conversation with my closest friends over a meal or a picnic in the park.</p>
<blockquote>
“We act as though comfort and luxury were the chief requirements of life. All that we need to make us happy is something to be enthusiastic about.” - Albert Einstein
</blockquote>
<p><br /></p>
<p>The overarching feeling is a sense of extreme thanks.</p>
<p>Extreme thanks for being at the right place at the right time. There’s no longer a feeling of looking for happiness (at least not consciously) or a goal to “be happy.” I feel like I am already there. The finish line is behind me, and everything I could have ever known to ask for, I already have.</p>
<hr />
One simple rule to fitness2015-03-04T00:00:00+00:00jasti.github.io/posts/everyday<p>Have you lost steam on your 2015 New Year’s resolution to get into better shape? I don’t believe in resolutions because I don’t think there is a bad time for making a good change. Having said that, I have a simple rule that I follow to stay fit (which is a very relative term).</p>
<p>It goes like this:</p>
<center><b>Workout. Every. Single. Day.</b></center>
<p><br />
You may be thinking that your body may not be able to handle the stress of working out every day. After some experimentation, I’ve realized my mind gives in waaay before my body does. Be it a workout or the ability to push through the days. The idea of giving your muscles and body rest is overrated. It began with bulky bodybuilders giving their body some rest after working out for about 5 hours a day. For the average being I am, my 1 hour per day workout routine is nowhere close to rest-day-worthy. The more I take my body outside its comfort zone, the more my body is willing to adapt.</p>
<p>The best analogy I can think of to compare working out to is flossing. Remember how hard it was when you started flossing for the first couple of times? Didn’t you want to quit? It was hard in the beginning to keep up with it everyday, but after a couple of weeks, you can’t not floss every day. The flossing analogy also works well because just like working out, they are both not mandatory (unlike brushing, which has immediate feedback) and the benefits or disadvantages are not realized until after some time has passed.</p>
<p>Sometimes things come up when mother calendar plays havoc, and I have a simple rule to accommodate that. If I miss a session, I have to workout twice the next day. Doing this makes me really think hard about missing my session for a day.</p>
<p>It is also important not to do the same workout everyday. Avoid monotony by switching it up with running, weightlifting, swimming, yoga and gymnastics.</p>
<p>The weekends can get hard, especially if you have been out Friday night, hanging out with friends. The thing that has worked for me is to commit to a Saturday morning workout session. In my case, I do a 90 min hot yoga class at 10 AM every Saturday. It kicks my butt every single time, but I know that I will feel that much better after the class, and what a way to kickoff the weekend!</p>
<p>We humans are built for doing hard physical things. Sitting at a desk, staring at a computer all day is unnatural. Make that commitment to Workout. Every. Single. Day and stop worrying about silly yearly fitness resolutions.</p>
<hr />
Learning New Programming Languages2015-01-12T00:00:00+00:00jasti.github.io/posts/learning<p>We spend about a third of our adult life at work. Consciously or not, a lot of our mental bandwidth is spent on thinking about future career options. If you are like me with a background in computer science, your default option is to become a software programmer. There seems to be an unlimited demand for programmers. Unless we see a repeat of the dot-com bubble pop, the demand for programmers will rise for the foreseeable future.</p>
<p>I began my career working for Goldman Sachs as a software programmer. I only get to spend about 50% of my time programming at work nowadays, but I need to learn new programming languages/ technologies all the time. In my experience, if you want to keep up with programming for the rest of your life, you have to be good at learning new programming languages.</p>
<p>My biggest insecurity during my first programming job was about my ability to sustain a lifelong career as a software programmer. I’m sure this question of being able to sustain a career for your lifetime is something almost everyone rattles with, but programmers may have it worse because of the rapid change in the software landscape. Especially the changes in the bread and butter of software - programming languages. Would I be able to keep up with these new languages as I got older?</p>
<p>Any high growth industry during its time is bound to such volatility in tools and procedures. Employees working in architecture and manufacturing fields in the 19th century probably felt the same way because there was constant change in tools and best practices until there was a clear winner in the way a new building was built or a new machine was manufactured.</p>
<p>Software development has an added dimension of keeping up with the continued rapid growth in computing hardware. Until the day <a href="http://en.wikipedia.org/wiki/Moore%27s_law">Moore’s law</a> becomes false and there is a clear winner in the way software ought to be developed, software developers have no way but to learn new programming languages. So how do we get better at learning new languages?</p>
<p>I’ve found that the biggest inhibition to learning a new language is <i>fear</i>.</p>
<p>Fear that a new programming language is harder than the one you already know. Fear that you are probably just wasting your time, because this new language will <a href="http://locklessinc.com/articles/why_lisp_failed/">fail</a> sometime in the future like a dozen other languages.</p>
<p>I am no expert in keeping up with all the new technologies, but I’ve been able to experiment my fair share with trending programming languages (you’ll know that a language is trending when you hear about it repeatedly on <a href="https://news.ycombinator.com/">Hacker News</a> or <a href="http://www.quora.com/Is-Go-golang-gaining-momentum">Quora</a>) over the years. Learning a new piece of programming language is similar to learning a new language, except for one major difference - the programming language is trending because it has endured the test of the programming community over the years. A corollary to a language’s popularity is that the language is usually <b>more programmer friendly than it’s predecessors</b>. As a result, it is easier to learn. The strategy that has worked for me is to waiting till a language reaches the plateau of productivity in the <a href="http://en.wikipedia.org/wiki/Hype_cycle">Gartner’s hype cycle</a>.</p>
<center><img src="/images/blog/newlang/Gartner_Hype_Cycle.png" /></center>
<p>Most languages don’t get this far, but once they do - you can be sure that the new language is not going to be more difficult than something you already know. <b>It’s just going to be different</b>. There is nothing left except for having the faith and attacking it by jumping into it with both feet. This observation is not earth shattering by any means, except that this realization negates the fears of wasted effort and self confidence.</p>
<p>Of course, we don’t need to learn every new language under the sun. The jury is still out on how much energy you need to spend learning something new vs sticking to what you know. It is a fine balance. This mostly depends on your available time and drive. It’s a topic that deserves a post of its own.</p>
<p>I’ve only discussed programming languages here, but it could very well apply to build systems, third party services, development environments, deployment solutions, etc. So the next time you are curious about learning a new piece of technology, remember to have the faith in the larger programming community to only promote the tools that make the developer ecosystem a <i>better</i> place. Take that first step. Believe that it is easier than the language you already know. Get your feet wet. Pick a good tutorial and hack your way through!</p>
<hr />
Hiking in Hawaii2015-01-03T00:00:00+00:00jasti.github.io/posts/hike<p>What is it about travel that makes most of us so happy? My best vacations have always been the ones where I end up in the places most different than my typical surrounding. Staying active on a vacation is important to me. In addition to rewiring your mental fabric and obvious physical benefits, it helps me not feel as guilty when I inevitably pig out on new grub.</p>
<p>Couple of weeks ago, Kristina and I decided to go to Hawaii on a 1 week notice. We booked our flights and we were off to Oahu and Kauai for 10 days. Our plan was to fly into Oahu, spend the night there, take a short flight over to Kauai for 7 days and then spend the last 3 days at Waikiki in Oahu. We heard from friends that Kauai was a hiker’s paradise. That is an understatement.</p>
<p>When booking your hotels using sites like Priceline, make sure you receive an email confirmation. We realized as soon as we landed in Hawaii, at about 9 PM, that Priceline failed to make our hotel reservation. We tried to make a reservation at the nearest hotel because we had a flight to Kauai at 8 AM the next day. As luck would have it, the Honolulu Marathon was scheduled for the next morning and EVERY single hotel was booked. We ended up renting a car for the night. Drove around the city, “watched” Interstellar for the second time (ended up dozing for the most part), got some late night food and walked around Waikiki between 3 and 5 AM. It turned out to be quite cool as we saw marathoners from all over the world going over their marathon rituals, making their way to the marathon start point.</p>
<p>In Kauai, there is a lot to do in the north and the south ends of the island, so it makes most sense to book your hotel some place in the middle of the island, like Kapa’a which allows you to drive up or down depending on what you want to do. It takes about 80 mins to drive from one end of the island to the other.</p>
<p>We set out to do the 8 mile loop to Hanakapi’ai falls and it turned out to be the most beautiful place I had ever hiked. The first half mile into the hike, you are greeted with this view.</p>
<center><img src="/images/blog/hike/napali_half_mile.jpg" /></center>
<p>I would love to carry around my DSLR for such trips, but they end up being so bulky that I tend to prefer my phone. The end result is not fancy, but gets the job done. I clicked this and all the pictures here with my Nexus 5 phone without any filters.
Standing there and looking far into the ocean makes you realize how far away from home you really are. I’m sure that what makes all of this so amazing is the novelty aspect.</p>
<p>Everywhere you look, the land gushed with greenery. Nature has a way of calming you down. Makes you more present. An escape from yourself.</p>
<p>Central Park is probably the greenest patch in Manhattan, and it has been landscaped by one of the <a href="http://www.fredericklawolmsted.com/central.html">best</a> landscapers ever, but it doesn’t even come half as close to this “random” shot on the hike in terms of plain beauty.</p>
<center><img src="/images/blog/hike/napali_nature.jpg" /></center>
<p>Hawaii is said to have a variety of flora and fauna that the rest of the world doesn’t have
<a href="http://en.wikipedia.org/wiki/Endemism_in_the_Hawaiian_Islands">because</a> of how remote the islands are and also because of the more relatively recent formation of the islands.<a href="https://www.khanacademy.org/science/cosmology-and-astronomy/earth-history-topic/plate-techtonics/v/hawaiian-islands-formation"> Here </a> is an excellent explanation in a Khan Academy video of how the islands were formed.</p>
<center><img src="/images/blog/hike/napali_flora_joined.jpeg" /></center>
<p>A lot of the trail was alternating between hard rock and extremely slippery mud. There were places in the trial where one wrong step or a skid could result in a 1000 foot fall. Some parts of the trail are maintained with heavy logs, so people didn’t slip and roll but some parts were totally wild wild west. My shoes didn’t thank me after the hike. We saw a guy on our way down who was hiking barefoot and we looked at each other and decided that the guy was completely mental.</p>
<center><img src="/images/blog/hike/napali_mud_joined.jpeg" /></center>
<p>A hike like this makes you wonder if fatalities occur all the time and you just don’t hear about them. A picture along the trail confirmed this.</p>
<center><img src="/images/blog/hike/napali_beach_warning.jpg" /></center>
<center><i> The bottom half of the warning illustrates the number of fatalities</i></center>
<p><br /></p>
<p>We also saw an action movie style rescue mission of a guy caught in a <a href="http://en.wikipedia.org/wiki/Rip_current">rip current</a> while swimming by the beach.</p>
<center><img src="/images/blog/hike/napali_rescue.jpg" /></center>
<p>You’ll find many places like the photo below where the water from the waterfall ends up with the ocean. Reminds you of your 5th grade science class when you learned about the <a href="http://en.wikipedia.org/wiki/Water_cycle"> water table</a>. A fun little challenge throughout the hike is crossing such rivers, hopping from stone to stone without getting your shoes wet. As you can imagine, easy to slip on those moss covered stones.</p>
<center><img src="/images/blog/hike/napali_river.jpg" /></center>
<p>At the end of 2 miles, you reach Hanakapi’ai beach which I hear is the listed as one of the most beautiful beaches in the world. The only way to get to it is by hiking the 2 miles because both ends of the beach are surrounded by giant hills that melt into the ocean.</p>
<center><img src="/images/blog/hike/napali_stones.jpg" /></center>
<p>As you go further up to the waterfall, you experience a ton more of the tropics, rivers to hop over, slippery mud and rainfalls that last for about 2 mins every 10 mins. We saw plenty of rainbows and some times even double rainbows.</p>
<p>We also spotted a wild kitty in the woods that was camouflaged nicely and a mouse that devoured cheese puffs on the ground.</p>
<center><img src="/images/blog/hike/napali_animals_joined.jpeg" /></center>
<p>After hiking for 2 more miles, you reach Hanakapi’ai Falls. The falls itself was nothing spectacular. Have seen better ones. All the more reinforcement that it is not about the destination but the journey.</p>
<center><img src="/images/blog/hike/napali_falls.jpg" /></center>
<p>We decided after this hike that we had to hike at least once a day in Kauai. On average, we were hiking about 6 to 8 miles and they were are all unique and amazing. If you ever plan on going to Kauai and want suggestions on stay, hike and food - <a href="/about.html">hit me up</a>.</p>
<p>We loved the Hanakapi’ai hike so much that that we went back to hike it again on our last day in Kauai. As we approached the hike, I was reminded of the guy who was hiking barefoot during our last visit. I didn’t think it was possible, but decided to give it a shot because of my recent obsession to differentiate between real limitations vs self-imposed limitations . This wouldn’t be one of my smartest decisions but was glad I went through with it.</p>
<p>I took a before and after expecting some significant damage but on the surface (the pun intended) my feet held up pretty well. The pictures in the middle show you parts of the trail made of dry sharp rocks and really muddy sharp rocks.</p>
<center><img src="/images/blog/hike/napali_feet_joined.jpeg" /></center>
<p>As it turns out, when you don’t wear any shoes, you start to notice the smallest pebbles on the ground. The lack of any grip doesn’t help in the slippery mud when going downhill. There were about ~50 times when I almost fell down and 2 times when I fell. You get used to the tiny knicks because of the sharp rocks after a point because your feet become numb. Hiking barefoot is a unique experience because you don’t spend too much time looking around you and instead mindfully focus on your next step. I won’t recommend going barefoot during your first hike on a trail, just because I wouldn’t want you to miss the incredible beauty around you. But if you have already hiked the trail before, why not have a more intimate experience with the ground beneath you?</p>
<p>If there is a paradise, I imagine it is like Kauai, but the thing is, you get used to it. At the end of 7 days, as much as I loved our time there, I was itching to go back. Among other things, was craving some intellectual stimulation. I’m sure that the grass is greener on the other side, but part of what makes these trips so amazing is the dosage they are served at. I didn’t believe that island fever applied to a place like Kauai but talking to the locals, they were far more interested in exploring New York City than staying in Kauai. I feel extremely fortunate and humbled by the opportunity to travel to these remote places, but, ironically, going away from home reminds you why you decided to pick your home where you did in the first place.</p>
<blockquote>
“The most obvious, ubiquitous, important realities are often the ones that are the hardest to see and talk about.“ - David Foster Wallace
</blockquote>
<p><br /></p>
<p><i>See the full essay for the quote above <a href="http://www.metastatic.org/text/This%20is%20Water.pdf">here</a>. </i></p>
<center><img src="/images/blog/hike/napali_final.jpg" /></center>
<hr />
Josie2014-11-30T00:00:00+00:00jasti.github.io/posts/josie<p><i>If you don’t care to read about pets, please stop and press CMD+Q on your keyboard. </i></p>
<p>Josie was a rescue. When Kitty-Kind rescued her, she was in an empty warehouse with a pile of dry food that somebody had left for her. Cats behave differently if you have had them since kittens. Rescues usually don’t trust their owners, and, especially in Josie’s case, have strong separation anxiety.</p>
<p>Three years ago, I became a pet owner by way of default. When I was about to ask Kristina to move in with me, my biggest hesitation was Josie. I never had a pet my entire life and quite honestly, didn’t see the point. Especially cats - you feed them, scoop their sandbox, arrange for cat sitters when you travel, frequently cleaning the house for cat fur… Sometimes it could be as simple as getting upset with her fur sticking on my formal pants.</p>
<p>Josie and I did not get off on a good start. I treated her like that roommate that never paid rent, ate the food we bought and made a mess of the house. Kristina and I rarely argue, and when we did, it was about Josie, because of the way we had to divvy up house chores. We also agreed that we weren’t going to allow Josie in the bedroom after we moved in together.</p>
<p>My relationship with Josie was transactional, and I always thought I couldn’t wait for the day until she was gone. Sure enough, I was dead wrong.</p>
<p>Over the years, Josie changed and so did I. Whenever one of us sat on the couch, she would come sit about 5 feet away from us. Expecting US to come pet her. When I cooked, she would come sit next to my feet praying dearly for a piece of meat to fall out of the wok. She would feel so lonely during the night that the moment we woke up, she would come into the bathroom with me and want pets. Once in a while when we slept in the living room instead of waking us with a meow, she would watch me from about 3 inches away from my face to see if I was awake. On the days we slept in our bedroom, she would be our wake up alarm as the sun rose. I used to get quite silly with her, by making her moonwalk to Michael Jackson’s tunes or by tumbling her on the couch. No matter what, she always came back to me for more. She was a fighter.</p>
<p>6 months ago, she was diagnosed with diabetes. Even just cutting Josie’s nails was quite a painful ordeal. I can’t imagine how much harder it would have been to give her insulin shots on her belly. We decided not to treat her because of the discomfort she and we would have had to go through. We made significant dietary changes and she recovered quite well for a while. It meant that we had to be more careful about how long we left her by herself, but that’s something K and I got used to over time.</p>
<p>Somewhere along the line, I don’t know when, the arbitrary, intangible relationship that I adopted by way of default turned into feeling completely responsible for her. I don’t know what it is like to have kids, but for me, helping Josie through her sickness bouts was the closest I have felt to fatherhood.</p>
<p>Last week, her hind legs gave away. It is a common side effect of diabetes. Apart from the fact that she had to delicately drag herself around the house, she also couldn’t get in and out of her sandbox anymore. She would get her front two feet in the sandbox, but unable to move her hind feet, she would pee on the floor next to her sandbox - thinking she was peeing in the sandbox.</p>
<p>From worrying about her fur sticking on my pants to not minding cleaning her pee outside her sandbox was quite a change in such a relatively short time. It’s amazing what you would do for someone once you are emotionally attached to them.</p>
<p>Kristina and I had decided that the day we felt like Josie was not having a good quality of life we would have to make the call. We tried out a couple of remedies, and she would only get worse. What would you do when you know that your kitty has only a day or two to live? The worst part of it all is the self doubt once you had made the call to put her to sleep. Every second, you look for the smallest improvements.</p>
<p>We went to the supermarket, bought chicken and there ensued my most emotionally-charged meal preparation in my life. Josie loved chicken. Anytime we prepared it, she would go around in circles in the kitchen, jumping up and down, until she got fed. I don’t know how cats look when they smile, but I know that the smell of fresh cooking chicken most definitely made her smile. Quite an irony that we were killing one animal to make another dying animal happy. I guess emotional bonds work in funny ways.</p>
<p>I don’t know when I’ll get used to not expecting her to greet us when we walk into the apartment or not expecting her to wake us up in the morning everyday or not always hanging around 5 feet away from us.</p>
<p>Non-blood relations can be like rollercoaster rides because they tend to leave you vulnerable when you least expect them, but with you, Josie, I would do it all over again.</p>
<p>I miss you my little girl.</p>
<center><img src="/images/blog/josie/josie.jpg" /></center>
Beyond Correlation Does Not Imply Causation2014-10-03T00:00:00+00:00jasti.github.io/posts/causation<center><img src="/images/blog/causation/correlation.png" /></center>
<p>This is my favorite xkcd cartoon. I didn’t <i>get</i> it immediately. If you spend a little bit of time, the strip is funny because the guy is not sure if the statistics class <i>caused</i> him to understand that ‘correlation does not imply causation.’ It’s quite meta, but once you understand that correlation does not imply causation, you tend to be quite cynical about all the articles in media that imply causation. For example, an article may conclude that ‘Companies that invest in technology tend to be more profitable than those that do not.’ But what if the investment in technology is not what is making the company profitable? What if the companies that are more profitable also tend to invest in technology? Therefore, the investment in technology does not necessarily cause profits. Maybe it does, but there is no one test that proves this 100%. Once this piece of insight is deeply ingrained in you, you stop taking for granted the claims and studies conducted in labs behind closed doors. It’s important to question almost every single article making a claim of causation that doesn’t use a combination of techniques to convince you of the claim.</p>
<h2>Averages and forces beyond measure </h2>
<p>If you know what short selling a stock means, skip to the next paragraph. A common strategy that traders use if they know that a stock is about to tank, is to short sell the stock. In reality, this means borrowing a stock from a lender for the current price at time t, sell in the market for the current price at time t, wait for the stock to nose dive between the time t and t+1, buy the stock back at time t+1 and sell it in the market at time t+1. Therefore, you fake sold the stock in the market at a higher price and bought the stock at a lower price, making the difference in profit.</p>
<p>Since short selling involves borrowing a stock, which costs money, many traders look at buying (going long) negatively correlated (the stock will drop in price when the price in the other stock rises) stocks instead. A lot of inexperienced traders get dinged because they assume that the stocks are negatively correlated and will stay negatively correlated during the time that <i>their</i> strategy is active. Primarily for two reasons:</p>
<p>Stock correlations are measured in averages over time. Traders usually have a time horizon to execute and reverse their trades to bank profit. So once you introduce the time variable (also known as a confounding variable) the negative correlation may not hold true anymore in the trader’s time horizon.</p>
<p>To understand the second reason, it is worth questioning why the stocks are negatively correlated in the first place. This could be due to a combination factors. One of them is due to large institutions having strategies that automatically rebalance their portfolio in a way that results in selling some stocks, when they buy certain stocks. This makes the market look like the stocks are negatively correlated. This works well and the day traders can use this to their advantage, until the time it doesn’t. Until the time the large institution changes its strategy or replaces one of the two stocks. This is just one example. Add a number of other variables to this, and it becomes hard to measure all the forces that impact this correlation.</p>
<h2>Can you prove that correlation is due to causation?</h2>
<p>The short answer is no but we can get comfortably close, depending on the business situation :</p>
<h3>A/B hypothesis testing with extreme sampling</h3>
<p>It’s quite a standard technique to take a wide variety of samples and run them through an A/B hypothesis test. It’s a relatively cheap mechanism to prove that your correlation hypothesis is indeed true. This is often done in web experiments to measure conversions, where a certain audience (Audience A) is exposed to a change and the remaining audience (Audience B) is not. The trick is to avoid as much <a href="http://en.wikipedia.org/wiki/Homophily"> homophily</a> as possible within a certain audience group. In reality, this would be quite hard to avoid and one solution is to assign the audience split using a perfect randomizer (i.e. extreme sampling).</p>
<h3>Eliminate the ‘B causes A’ and bidirectional fallacy</h3>
<p>When you observe that two entities are correlated and conclude that entity A causes entity B, it is also important to eliminate the other two causations. First, that entity B doesn’t cause entity A and second the causation is not bidirectional. For example, when one concludes that wind causes windmills to rotate faster, it is important to eliminate the possibility that a fast rotating windmill causes more wind. Also, we should eliminate the bi-directional causation aspect by highlighting that the wind is caused by the difference in heating temperatures over land vs water.</p>
<h3>Eliminate all possibilities of business knowledge</h3>
<p>An obvious way of eliminating correlations is to simply use experience over observations in data. Although, many managers of our generation take this too far. So it is important to reason why a causation is impossible when you see a correlation in the data. Here is an extreme example : There is correlation in data between the rise in ice cream sales and the high cases of drowning. This obviously makes no sense. The missing variable that the correlation doesn’t talk about is that these both occurrences happen only in the summer. It is natural that high weather temperatures cause ice cream sales to go up. Similarly with higher temperatures make more people go out swimming in open waters, causing more number of drowning related accidents.</p>
<h3>Granger’s causality test</h3>
<p>The Granger’s test is a little harder to implement. The basics of Granger’s causality test is to have enough experiments that will predict the future outcome of a time series based on past values on the time series. You can of course choose to run the experiment by back testing with data. Assuming it is time t now, you can use time t-2 to predict values at time t-1. In fact, this is a standard method used in predictive modeling to measure the predictive strength of a certain model - also known as holdout testing.</p>
<h2>Causal Modelling</h2>
<p>Even after performing all these tests, you can still be never a 100% sure that correlation does imply causation, but the good news is for a lot of the problems we face in business today we don’t require a 100% confidence.</p>
<p>It is up to the person making the claim to weigh the trade-off of increasing investment to reduce the assumptions made versus deciding that the conclusions are good enough given the assumptions.</p>
<p>There is a lot more theory around correlation and causation, but hopefully the next time you read an article that is making a causation claim, I hope you’ll be more cynical and ask for more backing of the claim instead of a lame correlation.</p>
Real World Big Data Techniques2014-09-10T00:00:00+00:00jasti.github.io/posts/bigdata<p>Big data has become a big buzzword. There are a ton of articles that come to the conclusion that big data = big profits for firms that are willing to use it, but don’t explain the <i>how</i> part.</p>
<p>Most articles explain solutions for big data problems by using basic statistics to analyze large amounts of data. By basic statistics I mean counts, means, medians, standard deviation. This is not new and a number of firms have been doing this since the late 90’s. This is called analytics. The difference today is the large amount of digital information being generated per second as well as the technology advances in software and hardware to tackle these large volumes of data.</p>
<p>In my opinion there are much more interesting uses of big data to uncover hidden potential and revenue-generating strategies of a firm’s business. And that is data mining / data science, a class of machine learning algorithms that are often overlooked in traditonal journalism.</p>
<center><img src="/images/blog/bigdata/ai.png" /></center>
<p>Machine learning is the ability to train a machine with data using algorithms.</p>
<p>It has been often <a href="http://anand.typepad.com/datawocky/2008/03/more-data-usual.html">discussed</a> that more data is better than a complex algorithm. My goal is to look at simple real world applications of data science using big data by employing different techniques, which addresses the <i>how</i> part of the big data = big profits equation.</p>
<p>In order to understand the techniques, it is important to understand what a model is.</p>
<p>A model is an entity that allows you to capture regularities in data. The way one constructs a model is by identifying attributes of a data set that are relevant to the analysis and use them (through feature extraction) to construct a model. If you are doing email analysis, your features could be sender location, email body text sentiment, etc.</p>
<center>
<br />
<blockquote>
Machine Learning Model = Data + Algorithm
</blockquote>
</center>
<p><br /></p>
<h3>Classification | Gmail Email</h3>
<p>Gmail (and most other email software) has the ability to detect if the email you received is from your friend or from that Nigerian teenager who is phishing (fishing?) with a new scam. To do this Gmail uses a technique called <b>Classification</b>. Gmail parsed all emails from the beginning of time and built a model. The next time a new email arrives, the features of the new email are input into the model in real-time and a decision is made to <i>classify</i> if the new incoming email is spam or good (ham?).</p>
<h3>Regression | Zillow Housing Prices</h3>
<p>Regression is used in situations where a system attempts to predict a numerical value. Zillow uses this by looking at other similar houses by building a model using features like number of bedrooms, location, school district, etc and estimates (Zillow calls this <a href="http://www.zillow.com/zestimate/">Zestimate</a>) the price of a house.</p>
<h3>Similarity Matching | Amazon Recommendations</h3>
<p>Similarity matching can be used to find similar entities of a product. We’ve all seen the “People who bought..” section on Amazon. The way Amazon makes these suggestion is by not just using empirical past transactions but also doing similarity matching of product features. Another interesting example is how IBM was successful in identifying similar firms, so it could put its salesforce to best use while selling a product.</p>
<h3>Clustering</h3>
<p>Clustering attempts to group entities together that are similar to each other but <i>not driven by any specific purpose </i>. I’ve found clustering useful during the initial phases of exploratory data analysis. Data scientists usually start with this algorithm to tackle ‘You don’t know what you don’t know’ scenearios.</p>
<h3>Data Reduction | Netflix Genre Prediction</h3>
<p>Netflix uses this as one of the many algorithms while trying to make genre recommendations for users based on large user movie watching history. This algorithm attempts to take a large set of data and replace it with smaller amount data which more or less contains the most important identifiable information in the larger data set. This is also known as noise reduction. Like clustering, this is also one of the first techniques applied when exploring data.</p>
<h3>Co-occurrence grouping | Walmart & Target Shelf Placement</h3>
<p>Co-occurrence grouping is used by by large retailers like Target and Walmart to place items on shelves next to each other. This is similar to clustering, but more specific. It allows to identifying items that were purchased in the same transaction. An obvious example is, if you bought gin, you are likely to buy tonic as well. <a href="http://www.forbes.com/forbes/1998/0406/6107128a.html">Beware</a> of the false positives you can uncover using this algorithm.</p>
<h3>Profiling | Chase Credit Card Fraud Detection</h3>
<p>Almost all banks like Bank of America, Chase, etc. have fraud detection on credit card purchases setup using profiling. Profiling is used to establish behavioural norms of a consumer. So in cases where an event occurs outside of the norm, the individual is alerted.</p>
<h3>Link Prediction | Linkedin People You May Know</h3>
<p>Linkedin is famous for introducing the <a href="http://data.linkedin.com/projects/pymk">‘people you may know’ feature</a>. A graph is constructed to form the relations between all the individuals (nodes), and if two people have a number (say 10) of common friends, they probably know each other.</p>
<p>That’s it for the whirlwind tour of big data using data science and goes without saying that one could write a book about each of these topics. My goal is to show real world applications of big data using machine learning and data analysis. The next time the journalist talks about “big data,” it might be interesting to consider not just the ‘what’ but the ‘how.’</p>
<hr />
Programmatically Downloading Legal Music2014-08-31T00:00:00+00:00jasti.github.io/posts/beats<p>For me, there are very few solo activities that don’t involve music playing in the background. Programming, writing, reading, working out and more recently, cooking. Although most of my music listening has moved online to a combination of Spotify, 8tracks and Pandora, there are a couple of reasons I still like to own my music offline.
</br>
First, if I am at a park or a place without Wi-Fi, I don’t want to be stranded without the music of my choice. Add to that the stupid data limits placed by phone networks which restrict your ability to stream music. Second, a lot of unique music is made by artists that don’t <a href="http://goo.gl/QFaaZ1">sell</a> their music, and therefore not available through Spotify type services.</p>
<p>A friend recently recommended <a href="http://musicforprogramming.net">MusicforProgramming.net</a>, and it is fantastic. The problem was that the music I wanted was sprawled across 27 pages on the site. I didn’t want to click on each link and download it manually. Moments like these are when I am truly thankful that I can code my way out such scenarios.</p>
<p>I wrote this little Python script that takes a website and downloads any mp3 files found on the site. Additionally, it will go to any links it finds on the given site, navigate to the pages and download all those mp3 files that it finds on each page as well. Since the files being downloaded are quite large, the script also prints the status of each download.</p>
<p>I’ve abstracted out the methods in such a way that the script can be reused for any website that hosts music, with minor changes. The primary functions in the script are:</p>
<ol>
<li>Given a page, find all the links in the page iteratively and collect all the mp3 file links</li>
<li>Given a mp3 file link, download it</li>
</ol>
<h2>Functions</h2>
<p>The function <i>findUrls</i> uses <a href="http://www.crummy.com/software/BeautifulSoup/">BeautifulSoup</a> to find all links on a page. This function is reused to grab all the URLs on a page and a second time to grab all the URLs on a page that end with the file extension ‘.mp3.’ You can obviously extend this to support any audio formats you like.</p>
<center><img src="/images/blog/beats/findUrl.png" /></center>
<p>The function <i>downloadMusic</i> takes an mp3 file as an argument and downloads the file to the folder from where the script is invoked. The file size is obtained from the header and the file is downloaded in chunks. The progress is output to the console in a while loop as the download progresses.</p>
<center><img src="/images/blog/beats/dlMusic.png" /></center>
<p>Last but not least is the main part of the script that ties it all together. The variable main_url is the URL where you want to download the music from and it function makes use of the other two functions above to fetch all the music.</p>
<center><img src="/images/blog/beats/main.png" /></center>
<h2>Running It</h2>
<p>You can find the full script in this <a href="https://gist.github.com/jasti/cf5188b411d185a4f27a">gist</a>. Run this from the command line by simply typing ‘python Beats.py’. You will see the files getting downloaded to the same location you ran it from on the disk.</p>
<center><img src="/images/blog/beats/running.png" /></center>
<p>Feel free to copy, download, share or distribute. You should be able to reuse this by simply editing some variables. For e.g., I programmatically download some of my favourite audio podcasts from KEXP’s <a href="http://feeds.kexp.org/kexp/musicthatmatters">‘Music That Matters’</a> that I often use for my long runs.
</br>
HUGE time-saver. More time for watching those cat videos, you know.</p>
<hr />
Client Solutions In Enterprise Software2014-08-07T00:00:00+00:00jasti.github.io/posts/solutions<p>You are 3 months into building the MVP for your enterprise software startup. Your product is ready for launch. You have had the time to hire a few sales folks to go out and sell your product. The product is quite successful and a bunch of clients sign up for the product. How you onboard your client can be make or break for the long term relationship with the client. A lot of early stage companies focus on sales, but the secret sauce for long term growth and retention is implementation of the product. Obviously, you can’t have the later without the former, but I want to highlight why having a client solutions function can sometimes be as important as (if not more than) having a sales function.</p>
<p>Enterprise software (think customer service software, hedge fund trading software) is different than commercial software (Kayak, Airbnb, Facebook) in usage, complexity and price. There are fewer users using enterprise software, functionality is more rich, and product margins are higher. In contrast to enterprise software, you don’t see a huge need for sales and client solutions in the commercial software business. Competition to sell an enterprise product is a lot more cut-throat because deals are much bigger. Enterprise software is built to be more configurable because of the ability to satisfy niche workflows and use-cases.</p>
<p>A lot of pure-bred software engineers have asked me the value of client solutions. Shoudn’t software “just work”? Here are a couple of reasons why enterprise software companies must provide client solutions right after you start selling the product:</p>
<h2>Deriving Value</h2>
<p>If the software configuration is left to the client, there is a high likelihood that the client uses the product in unintended ways, and worse, only uses some functionality and ignores rest. Chances are, the client won’t derive the full intended value from the product. This is more common than one would think. When economic times are good, enterprise executives are wallet-happy. Especially during times like right now, when interest rates are dirt low and the COO expects the value derived from the software to be higher than the IRR from placing that money in other investments. Executives get easily impressed by the sales demo and agree to buy the software, without fully understanding implementation and use-cases.</p>
<p>For example, often implementation is left to the head of IT at the client. An important motivation for the head of IT is saving infrastructure costs. Since he is not an end user of the product, he decides to allocate a single machine instead of load balancing between two machines. Product is not as snappy and users don’t know to describe the problem or know that the product could be much faster. End result, users shy away from using the product. Everything stays status quo until the economy turns downhill, and executives look for any and all ways to reduce expenses. Your software is not being used the way you wanted it to be and therefore not sticky. As a result, the client wants out.</p>
<p>The best way to get around such problems is to build a client solutions team and hire a few really good ‘consultants’ (who are really just employees of the startup) to take responsibility of implementing the software after sales sells. Consultants implement the software the way it was meant to be used and consistently work with the client to derive value. This makes software sticky. Client executives see the real value and see the point of their investment even when times are bad.</p>
<h2>Product Feedback</h2>
<p>Traditionally, product management populates the feature backlog and sets the roadmap for features to come. Having your consultant implement and maintain a client relationships helps gain continuous feedback like nothing else. This constant feedback helps hold a pulse on client satisfaction. It helps prioritize the features that are being requested by different clients instead of simply predicting or relying on clients that spend the most.</p>
<p>Firms usually have more than a single product offering and in cases where your consultant is able to monitor the client’s satisfaction, he is able to cross sell other products, allowing further stickiness.</p>
<h2>Fixing The Sales Demo Hangover </h2>
<p>Sales is a hard job and sales engineers do everything in their power to control the things they can during a demo. More often than not, client will see a stellar demo but the cobwebs don’t get exposed until the client starts using it. The devil always lies in the details. Sales does a fantastic job with sandboxed data but come implementation time, the product doesn’t suit all possible use-cases. Almost no product ever does. The client is unhappy even before the product is live!</p>
<p>Instead, by assigning a consultant to the implementation, he is able to provide most value by building workarounds where the product doesn’t completely satisfy the use-case, maintaining the client relationship and best of all, channeling these edge use-cases back into the product. If those features get added to the next sprint release, it is a win-win situation for both the client and the product.</p>
<h2> Configuration vs Customization </h2>
<p>Designing a product to be highly configurable is obviously very good, but the configuration framework (things like rules based database or locale settings) won’t be able to handle all the different use-cases. The solution is to design a product that is highly customizable, but it can be a double-edged sword. Consultants are able to accommodate use cases / workflows by building custom solutions. Unless you end up incorporating all these customizations back into the product in a timely manner, upgrading the client to the next version will be very hard.</p>
<p>In early stages of high growth start ups, prioritization of features is extremely important. It is probably the most important thing that makes or breaks a product. Having a consultant to build workarounds and customizations allows the product team to focus on the absolutely necessary features or bug fixes that have broader applicability for the product. This buys time with clients who can’t be immediately accommodated by the product team.</p>
<p><br />
The downside of having a client solutions department is obvious. There is an initial upfront capital expense. The size of the department needs to scale with the deals in the sales pipeline. It is quite hard (and often expensive) to hire consultants who have good functional knowledge, are good at customer service and have the technical capability to build workarounds and customizations in the product. One can’t make too many mistakes with these hires because quality of service directly correlates with client retention and these hires are the face of the product.</p>
<p><br />
There are so many things that can’t be controlled by a company, including market conditions, competition, and clients going out of business. The one important thing that a company <i>can</i> control is the implementation, and there is no better way than to build a client solutions organization to excel at that.</p>
<hr />
Getting Weird In Austin2014-08-01T00:00:00+00:00jasti.github.io/posts/austin<p>Kristina and I got married two weeks ago. If you aren’t married already, the first question someone asks you after they hear that you got married is, ‘where is the honeymoon?’ I don’t know if people are genuinely interested or making conversation or are on auto-pilot. I want to believe that majority fall in the 3rd category. Honeymoons were introduced way back when so couples could get personal in an intimate setting, easing them into a physical relationship. Times have obviously changed. Similar to political lobbying, it doesn’t make sense, is expensive and the participants are playing with antiquated rules. Time to move on…</p>
<p>We didn’t have a honeymoon, but we did make a trip out to Austin to visit some family over the weekend. We both fell completely in love with it. Austin was different and it was unlike any other place I have traveled to in the US. I dare say that it was one of the very few cities (other than New York & San Francisco) that showed quite a bit of character.</p>
<p>The first thing as a tourist you might notice are the signs which say ‘Keep Austin Weird’. It is true. I think people take a uncoordinated under the table approach to keeping Austin weird and hence different. For example, almost all the bars have a stage for a music performance. Pets are treated as first class citizens. One bar we went to had a pet play area, too!</p>
<p>Having lived in New York city for quite a bit, you tend to have your guard up all the time. Feeling insecure and always alert. You feel like someone is always out to get you. Austin is quite the opposite of that. People want to help you and if you are like me, you will look down and scurry when someone greets you with a pleasant ‘good morning’ on the jogging path. You get used it soon enough, though. People have a laid back approach to things and seem to have a very ‘live and let live’ mentality.</p>
<p>The downsides are the heat and Austin’s underwhelming coffee shop selection.</p>
<p>There is outdoor seating almost everywhere you go with large fans that spray mist which helps with the ridiculous heat. We were there on one of the hottest days of the summer and it is impossible to sit outside without these misting fans. We tried about 5 coffee shops and had a rather OK experience. From a free-willed place like Austin, my expectations for coffee were much higher. Maybe we didn’t hit the right ones.</p>
<p>*** UPDATE : Take my statement back about coffee shops after discovering Caffe Medici!</p>
<p>Here are some highlights from our trip, so the next time you tell me that you are thinking about a trip down south, I can direct you to this post.</p>
<h2>Do</h2>
<table>
<tbody>
<tr>
<td>Zilker Park</td>
<td>We did a great 6 mile run around the Ladybird Lake. Highly recommend.<br /></td>
</tr>
<tr>
<td>Barton Springs Pool</td>
<td>A great natural pool fed from underground springs. I had never seen anything like this before.<br /></td>
</tr>
<tr>
<td>Hamilton Pool Preserve</td>
<td>This reminded me of Avatar, with the mountains in the sky - but in reverse.<br /></td>
</tr>
<tr>
<td>Mt Bonnell</td>
<td>Beautiful view of Austin. You realize that Austin has a TON of water pockets in and around.<br /></td>
</tr>
<tr>
<td>Alamo Drafthouse</td>
<td>It is like NiteHawk in Brooklyn, but this is the mothership. More space, better food.<br /></td>
</tr>
<tr>
<td>South Congress</td>
<td>You can’t miss it. Bum around, antique stores, trailer food & music.<br /></td>
</tr>
</tbody>
</table>
<h2>Eat</h2>
<p>Hopdoddy| THE best vegetarian burger ever. Couldn’t resist going back a couple of times.<br />
Torchy’s Tacos| Damn good tacos. Try the fried avocado taco & corn.<br />
Fonda San Miguel| Little out of the way and expensive but utterly worth it. Had the best mexican food (not tex-mex) we had ever tasted.<br />
Trailer Food in East Austin| You’ll probably see this everywhere, but be sure to check it out in East Austin. Plus East Austin is a cool neighbourhood to walk around.<br /></p>
<h2>Drink</h2>
<p>Rainey Street| A cool little street with a ton of bars. Outdoor seating and almost every bar is unique.<br />
6th Street| Didn’t blow us away but a ton of bars. Worth walking down once.<br />
Easy Tiger| Cool bar on 6th street which doubled as a bakery. Has ping-pong!<br />
East Side Showroom| Throwback bar with excellent cocktails.<br /></p>
<hr />
A Case For Tech Optimism2014-06-23T00:00:00+00:00jasti.github.io/posts/robot<p>It is only a matter of time that <a href="https://www.youtube.com/watch?v=CqSDWoAhvLU">Driverless Cars</a> will obviate all drivers - taxi, bus and truck. This will not be at the scope of the agricultural or industrial revolution, but close. Many will lose their jobs. Controversially, that may be a good thing.</p>
<p>You have probably noticed the outcry in the media about how robots are eating our jobs. The media is right about the statement but wrong about the sentiment. Progress is impossible without disruption. Hindsight is 20/20 and you’ll find very few people who believe that the industrial revolution was a bad thing, even though it caused many hard working manual workers to lose their jobs. The standard of living <a href="http://www.econlib.org/library/Enc/IndustrialRevolutionandtheStandardofLiving.html">rose</a> over the next 100 years, and the life expectancy increased significantly from approximately 40-45 years to 80 years in the United States.</p>
<p>Internet was first used only in academia and defence. As the network effects piled up, existing industries were disrupted. Just how Amazon.com is disrupting the brick and mortar stores, not long from now, we will see disruption in the physical manufacturing space with <a href="http://en.wikipedia.org/wiki/Oculus_VR">Oculus Virtual Reality (VR) </a>. VR will do to a lot of physical devices what the iPhone did to physical keyboards on smart phones with touch-based keyboards.</p>
<p>Oculus VR has come closest to making the virtual reality dream possible with their first offering - Oculus Rift. The company is going after the 3D gaming space first, but the applications for the Rift are endless.
Keyboard and mouse signals will be integrated to the VR software and you wouldn’t need physical computer monitors anymore. With cloud connectivity, the notion of a physical computer will be gone. Your software and data will be able to follow you. <a href="https://www.youtube.com/watch?v=Dgtba_GpG-U">Here</a> is a cool demo of how the future of programming could be like.</p>
<p>Oculus Rift could disrupt the entire film industry by giving everyone their own personal 3D IMAX theater. This could mean that the televisions could become obsolete as well. For a lot of us going to the movies or watching TV is a social experience. Some people think that technology and the internetization (it is a real word now) of things removes the human element. This is far from the truth. Technology allows humans to make a connection which couldn’t have happened without the technology. Personally, it would have been impossible to keep in touch with all my friends back in India without Facebook. Facebook didn’t buy Oculus VR just for it’s cool technology. VR will fundamentally change how people stay in touch.
With Oculus Rift, there is nothing stopping you from still hanging out with your friends. Instead of going to a certain place to do something, you and your friends could hang at a park and watch the same movie on your devices.</p>
<p>Technology innovations are adopted in the mass market because of a combination of three reasons - price, convenience and quality. The next iteration of VR promises all three, which is a quite rare.</p>
<p>Voice technology (VT) is getting started. Not long from now, VT will be able to detect <a href="https://www.youtube.com/watch?v=yR0lWICH3rY"></a> accents and reliably understand us. Andriod and Siri have shown what is possible so far, but it is just the beginning. Anything you touch today will turn into voice commands as input to a device. Possibly your phone, your watch or your <a href="https://www.google.com/glass/start/">glasses</a>. From keyboards to light switches in your living room.</p>
<p>Vision is not widely discussed in the market yet. <a href="https://developer.amazon.com/public/solutions/devices/fire-phone/docs/understanding-firefly">Amazon Firefly</a> seems to be quite reliable with initial testing. Harder than image recognition is moving pattern recognition. The driverless car has a 360 degree camera and is able to understand its surroundings using pattern matching and probability. By the time legislation approves the use of driveless cars by public, more use cases for utilizing the vision technology will go main stream.</p>
<p>Large number of industries will be disrupted when voice and vision technologies get better.</p>
<p>Robots will eventually eat <a href="https://www.youtube.com/watch?v=8_lfxPI5ObM">all</a> manual labor jobs. Although AI hasn’t got to the point where it can replace knowledge workers yet, it seems probable that it is on track within the next 20-30 years. It is not as morose as it sounds though. Humans have always found new ways to occupy themselves and we as consumers always <i>need</i> new things. Tech innovation results in disruption but only a temporary disruption amongst workers and shift in capital.</p>
<p>I’m optimistic about such transformation because once we get the robots to deal with the fundamental day-to-day problems, we are able to raise our head above our desks and explore more important and interesting problems. Solving world hunger (for real), space exploration, exploring the reason for existence, universal healthcare - to name a few.</p>
One Thing About Meditation2014-06-15T00:00:00+00:00jasti.github.io/posts/meditate<p>We have all heard about how meditation is good for you. The benefits include peace of mind, happiness, stress-relief and a better quality of life among other things. Having said that, there is still a lot of mystery around the connection between the physical action of meditation and its benefits.</p>
<p>Having practiced a couple of different techniques and thinking about my personal experience with meditation, it comes down to this: mind training. Mind training allows you to execute after you have decided to do something. Think about it. You are not the same thing as your mind. You know what you should be doing but your mind has a plan of its own.
How many times have you decided on working on a side project or cleaning up your apartment, but ended up watching YouTube videos instead? It takes immense amount of discipline to stick to doing something after deciding to do it. Sometimes this discipline can come in the form of deadlines enforced by external circumstances.</p>
<p>Meditation creates an environment that establishes the hierarchy of control between you and your mind. By stilling your mind and focusing on your breath, you are training your mind to listen you. When time comes for decisions and actions, the training enables you to do what you really should be doing. No more excuses for not going to the gym or not eating healthy or not developing yourself personally.
I have a simple technique for meditation. Count from 1 to 8 in one long exhale and count back from 8 to 1 in one long inhale. Repeat this for the duration of your meditation. The counting can be mississippi-fully, if it helps you :). If you occupy your mind to count while you breathe, it is impossible to think of anything else. It’s a simple way to occupy your mind to do what you want it to do.</p>
<p>I am nowhere close to being able to fully control my mind, but after meditating for a couple of months now, I am more disciplined than I used to be. Maybe it is just “placebo,” but it is a placebo that works. I suggest starting small. My fiancé and I meditate every night for 7 minutes before going to bed. It is amazing how hard it is to stay idle without doing anything for such a short period. It reminds us every day how easily your mind can own you. I have a long way to go but doing this consistently over the last couple of months has made a world of difference. I look forward to the journey.</p>
<p>I’d be happy to answer any questions you have. If you have never meditated before, I suggest that you start with this excellent book called <a href="http://www.amazon.com/Why-Meditate-Working-Thoughts-Emotions/dp/1401926630">Why Meditate</a> by Matthieu Ricard.</p>
<blockquote>
“You must be shapeless, formless, like water. When you pour water in a cup, it becomes the cup. When you pour water in a bottle, it becomes the bottle. When you pour water in a teapot, it becomes the teapot. Water can drip and it can crash. Become like water, my friend.” - <b> Bruce Lee </b>
</blockquote>
<hr />
Data Analysis Of Crossfit.com Workouts2014-06-08T00:00:00+00:00jasti.github.io/posts/analysis<p>Exploring data without knowing what you are looking for allows you to find results that you didn’t know existed. Below is a chart from business school 101. The top right quandrant is seldom explored by companies that are not established as data-driven companies.</p>
<center><img src="/images/blog/analysis/Unknown-unknown.png" /></center>
<p>The interwebs does a great job of explaining concepts, but not so much of explaining approaches to solve a <i> type</i> of problem. My goal is to show you how I approach exploratory data analysis problems and leave you with a set of strategies and tools. At the very least, it will help you get started in the right direction.</p>
<h2>Finding data to explore</h2>
<p>The ongoing data revolution has driven plenty of organizations to make their data available publicly. A few examples are <a href="https://www.data.gov/">here</a>, <a href="https://nycopendata.socrata.com/">here</a>, <a href="http://enigma.io/">here</a> and <a href="http://www.kdnuggets.com/datasets/index.html">here</a>.</p>
<p>For this post, let’s pick data that we previously scraped in <a href="http://vamseejasti.com/posts/scraping/">this</a> post. If you haven’t read it already, I recommend you do - for context. We ended up with a tab separated file which has all the workouts scraped from the <a href="http://crossfit.com/">Crossfit.com</a> website from the year 2002 to 2013. You can find a link to the file <a href="https://github.com/jasti/CrossfitAnalyser/blob/master/input/workouts.tsv">here</a>.</p>
<p><i>Sidenote : My liking Crossfit has got nothing to do with this dataset. OK, maybe it does. A little. I wanted a practical application and this data was handy. That’s it.</i></p>
<p>Before I wrote this post, I had no idea what I wanted from this analysis. I simply felt that there was something hidden in the data.</p>
<h2>Picking your tools</h2>
<p>The data science / analytics space is in such flux that there are no tools that are clear winners yet. ‘R’ used to be the go-to data analysis environment. Although there are plenty of packages available for R, Python is flexible across platforms, has a great ecosystem, many applications and a huge selection of libraries.</p>
<p>For this analysis, we’ll use the following tools:</p>
<ol>
<li>Python</li>
<li>Pandas</li>
<li>Open Refine</li>
<li>IPython notebook</li>
</ol>
<p>I have a brief description and installation instructions for tools 1, 2 and 4 in the my previous post <a href="http://vamseejasti.com/posts/scraping/">here</a>. Open Refine is a UI driven open source project that allows you to clean up messy data, especially the type of data you get from web scraping.
You can find installation instructions <a href="https://github.com/OpenRefine/OpenRefine/wiki/Installation-Instructions">here</a>.</p>
<h2>Viewing your data</h2>
<p>Although the data can be viewed in a plain Notepad or Excel, you want a tool in which you can experiment and immediately see the results. This is where the IPython Notebook shines. It is an interactive web interface that allows you to interface with your data, one step at a time.</p>
<p><i> I’ll have all this code available on Github, along with a sample IPython Notebook, so just follow along.</i></p>
<center><img src="/images/blog/analysis/01-view-data.png" /></center>
<h2>Sanitize your data</h2>
<p>I use Open Refine to deal with all sought of messy data. The usual suspects are:</p>
<ol>
<li>Leading or trailing white spaces</li>
<li>Control characters</li>
<li>Converting all your text to a single case</li>
<li>Removing duplicates</li>
</ol>
<p>Looking at the example below, here are two phrases that mean the same thing, but could be typed in many different ways:</p>
<center><img src="/images/blog/analysis/02-differences.png" /></center>
<p>Open Refine does a good job of combining these type of phrases into one representation. The primary concept used under the hood is called <a href="https://github.com/OpenRefine/OpenRefine/wiki/Clustering-In-Depth#key-collision-methods">Key Collision</a>. If you want to understand one way to apply a key collision method, fingerprinting is the most basic of all. The <a href="https://github.com/OpenRefine/OpenRefine/wiki/Clustering-In-Depth#fingerprint">Open Refine docs</a> do a great job of explaining it.</p>
<p>In order to get your data into Open Refine, let’s export the data from Pandas in a CSV format.</p>
<center><img src="/images/blog/analysis/02-1-open-refine-export.png" /></center>
<p>Import the file you exported from Pandas into Open Refine</p>
<center><img src="/images/blog/analysis/03-open-refine-import.png" /></center>
<p>Cluster all the workouts, which will give you the option to merge the similar data using the fingerprinting method.</p>
<center><img src="/images/blog/analysis/04-cluster.png" /></center>
<p>Open Refine has other methods to merge this sort of data and the option you pick depends on the problem at hand. For our case, the ‘fingerprint’ and ‘ngram-fingerprint’ techniques work quite well.</p>
<p>The tool forces you to go over each cluster it finds to merge, which is great. We only want to merge items that truly mean the same thing.</p>
<p>Next, clean up your data for the usual suspects:</p>
<center><img src="/images/blog/analysis/05-common-transformations.png" /></center>
<h2>Creating Derived Data</h2>
<p>When you have data in one column that needs to be split into multiple columns or vice-versa, depending on the problem, you can either use Open Refine or Pandas.</p>
<p>Some of the workouts on Crossfit.com are quite hard. These are usually Hero WODs (WOD stands for Workout of the day) or Benchmark WODs. The Hero WODs are named after fallen soldiers in war and benchmark workouts are have girl names and allow you to benchmark yourself over time. I wanted to see if there is an variance in the number of such workouts, year over year.</p>
<p>I noticed that all hero and benchmark workouts begin with double quotes. I created a derived boolean column out of the workout columns using the GREL syntax as below :</p>
<center><img src="/images/blog/analysis/06-grel.png" /></center>
<p>Now let’s import the data back into Pandas to generate some more derived columns.</p>
<center><img src="/images/blog/analysis/07-import.png" /></center>
<p>The date column is not very useful in this case if we want to perform any time series analysis. So let’s go ahead and split that into Year, Month and Day using Pandas.</p>
<center><img src="/images/blog/analysis/08-splitDate.png" /></center>
<p>Now our data is starting to look a little better.</p>
<center><img src="/images/blog/analysis/09-splitDateResult.png" /></center>
<p>As you can see, the possibilities to derive and combine columns are endless.</p>
<h2> Perform Sanity Checks </h2>
<p>When dealing with unstructured data, like data scrapped from the web, it’s important to perform basic sanity checks. Especially with time series data, you want to make sure you have data for all time buckets. Let’s ‘Group By’ year to get the workout counts:</p>
<center><img src="/images/blog/analysis/10-groupBy.png" /></center>
<p>Let’s create a simple histogram with year buckets to ensure we have all the data points.</p>
<center><img src="/images/blog/analysis/11-histogram.png" /></center>
<p>As you can see, we have fewer workouts for the years 2003, 2004, 2006 - so let’s get rid of that data from our dataframe.</p>
<center><img src="/images/blog/analysis/12-filter.png" /></center>
<p>Real world data is messy and uncomfortable. Start getting used to it and find ways to normalize data that is available. We will ignore this step for this post, to keep it simple.</p>
<h2>Analyzing your Data </h2>
<p>Here is where the real fun begins. You have clean data and all the columns you need. Let’s ask some interesting questions:</p>
<h3> Has there been an increase or decrease in the number of 'hard' workouts? </h3>
<center><img src="/images/blog/analysis/13-hero.png" /></center>
<p>The direct answer to the question is neither. There was a peak in 2008 and ever since, the number trended down until 2011 and then picked up again in 2012.</p>
<p>The workouts shown in 2002 are outliers. They just happened to be workouts that started with double quotes. Ignoring that, we can tell that the Hero workouts / Benchmark workouts first started to appear in 2005 in our data. Going back and looking at the years of data we ignored (because of incomplete data), the first Hero WOD was posted on the 15th of July on 2004.</p>
<p>Comparing this data with the number of fallen soldiers (below) in the Iraq war, you notice a laggard correlation between the two data sets. Especially the data between 2007 and 2009.</p>
<center><img src="/images/blog/analysis/14-iraq.png" /></center>
<p><i> Source : Wikipedia </i></p>
<p>This could very well be a case where we are looking too closely for an answer and there might be no relation at all. If you force data to find relations, you will most probably be able to do it - but it’ll be wrong. This concept translates nicely to the concept of overfitting a model in machine learning but let’s dive into that another day.</p>
<h3> With Crossfit's popularity, have there been fewer rest days programmed every year? </h3>
<p>We can solve this problem for the occurrence of the keyword ‘Rest Days’ in the workout column. We can bucket these occurrences by year and see a trend over time.</p>
<p>Let’s write a quick function that returns a series of the occurrence of a term in a data frame, grouped by year.</p>
<center><img src="/images/blog/analysis/15-search.png" /></center>
<p>Using this function, we can pass in the search term to see the occurrences.</p>
<center><img src="/images/blog/analysis/16-rest.png" /></center>
<p>I was surprised to see that there was no real differences in the number of rest days programmed per year. The rest day schedule has been quite consistent and the programming itself is not completely ‘random’.</p>
<h3> How many workouts have "cardio" involved? What is the trend? </h3>
<p>Traditional cardio can be running, swimming, rowing and biking. We can make use of the previous search function we wrote to tabulate the occurrences of corresponding cardio exercises.</p>
<center><img src="/images/blog/analysis/17-cardio.png" /></center>
<p>We concatenate all these dataframes into one dataframe which allows us to represent them as a stacked bar. A stacked bar is quite useful in comparing items placed in a time series.</p>
<center><img src="/images/blog/analysis/18-cardio-bar.png" /></center>
<p>There are a couple of neat observations we can make from this bar chart:</p>
<ol>
<li>Year 2002 was when rowing was first becoming popular and clearly the excitement is apparent in the workouts as well.</li>
<li>Swimming was never part of the workouts until the year 2011.</li>
<li>Biking was huge in 2002 and had completely died out by 2009, but is making a return again.</li>
<li>There has been a steady decrease in the number of running workouts and a steady increasing of rowing workouts again.</li>
</ol>
<p>The list goes on…</p>
<p>A lot of different questions can be answered with this dataset and all the techniques we covered here. If you have interesting questions or get stuck trying to solve for a question, just let me know and I will be happy to assist.</p>
<p>There are a lot of unknown unknowns out there and exploratory data analysis allows you to make them known knowns.
Don’t be afraid to explore them!</p>
<p>I love exploring data. Nothing explains it more than this quote.</p>
<blockquote>
“Fall in love with some activity, and do it! Nobody ever figures out what life is all about, and it doesn't matter. Explore the world. Nearly everything is really interesting if you go into it deeply enough. Work as hard and as much as you want to on the things you like to do the best. - <b>Richard Feynman</b>
</blockquote>
<p><br /></p>
<p>As always, you can find the code and the IPython Notebook on Github <a href="https://github.com/jasti/CrossfitAnalyser">here</a>.</p>
<hr />
Big-O Analysis For Recreational Programmers2014-05-29T00:00:00+00:00jasti.github.io/posts/bigo<p>Part of why I write these technical posts is to get a better understanding of a topic. You ask why I can’t do this without writing it as as a blog post? It’s like when you clean your house when you know you have guests coming. You could clean all the time, but you don’t.</p>
<p>Computing theory is hard to grasp unless it is laid out in concrete examples. I had a tough time understanding Big-O analysis in school. A self taught programmer who I met at <a href="http://hackerhours.org/">Hacker Hours </a> asked to explain big-O analysis and I asked him for a week to write this post. So here is my attempt at demystifying it - in plain English. This is not a theoretical post. I don’t claim to know all the theory behind big-O analysis. This is simply a way of understanding the fundamentals without getting bogged down by academic lingo, formulae and greek letters.</p>
<h2> Performance analysis vs big-O analysis </h2>
<p>We have all heard about performance. It is the amount of time taken by a certain task. The task could consist of a single operation or a hundred operations. The performance could depend on a lot of different factors like the machine we run it on, the programming language environment, network latency, etc.</p>
<p>Big-O analysis is not the same as performance analysis. Big-O analysis is useful in figuring out the best algorithm for a task at hand. But how do we define “best algorithm”? And what is an algorithm anyway?</p>
<p>Simply put, an algorithm is a <i>way</i> to solve a problem. Big-O analysis will help us understand how well an algorithm will scale in the worst-case scenario. This allows us to compare two different algorithms solving the same problem.</p>
<p>Reasonable parameters to consider if the algorithm is good can be measured with two factors :</p>
<ol>
<li>How much time is spent performing operations? (We will define an ‘operation’ below). Also known as time complexity.</li>
<li>How much memory does it occupy on a machine? Also known as space complexity.</li>
</ol>
<p>We will avoid considering the space complexity for this discussion and as do most people when talking about big-O analysis.In the rest of my post, complexity == time complexity.</p>
<p>In a loose sense, performance is useful in measuring absolute time spent on a certain task. However, big-O analysis is about relative time spent by the algorithm to solve the problem, thus leveling the playing field.</p>
<p>Let’s say we are given some unsorted numbers and we need to arrange them in ascending order. As you probably already know, there are many ways to <a href="http://en.wikipedia.org/wiki/Sorting_algorithm#Popular_sorting_algorithms">sort</a> those numbers. Let’s look at two simple sorting techniques, sorting the same input numbers : <br />
<i> If you just arrived to the show below, simply refresh the page and it will start from the beginning :) </i> <br /></p>
<h3>Merge Sort </h3>
<center><img src="/images/blog/bigo/Merge_sort.gif" /></center>
<h3>Insertion Sort </h3>
<center><img src="/images/blog/bigo/Insertion_sort.gif" /></center>
<p><i> Animation source: Wikipedia </i> <br /></p>
<p>As we can see, the number of operations performed on the set of numbers is different in each case. Depending on the input sequences and the size of these numbers, one algorithm <i>relatively</i> performs better than the other.</p>
<h2>Representation</h2>
<p>The output to a big-O analysis is always, surprise, a big ‘O’ followed by parentheses. Within the parentheses is expression in terms of ‘n’ or the number 1.</p>
<p>What goes into the parentheses is determined by the factors below.</p>
<h2> Big ideas of big-O analysis </h2>
<p>There are 3 main ideas we need to think about when performing big-O analysis of an algorithm:</p>
<h3>Input</h3>
<p>The input is directly correlated to the complexity. Depending on what the input is and the operations we need to perform on it, the complexity of the algorithm can be different in the best, worst and average case.
For example, in case of the sorting a set of unsorted numbers, the complexity of the algorithm will depend on the size of the input. The best case scenario for any sorting problem is to have an input that is already sorted. In the animations above, notice how insertion sort will be faster than merge sort in the best case scenario because merge sort will need to divide the list of numbers into a size of 1, regardless. Insertion sort on the other hand will make only a single pass through all the elements.
Similarly, the complexity might vary in the best or average case for different algorithms depending on how the algorithm is implemented.</p>
<h3>Operation</h3>
<p>An operation could be an iteration of a for-loop or could be as simple as returning the first element of a list, regardless of the size of the list.
In case of the for-loop, if the number of iterations we perform is the same as the size of the input, the algorithm is said to have a complexity of O(n). In the case where the operation returns the first element of the input, the size of the input doesn’t matter. The number of operations performed is always 1. This is known as constant complexity and represented as O(1).</p>
<h3>Ignoring lower order terms</h3>
<p>When performing big-O analysis, we ignore the lower order terms and constants. Consider an algorithm that performs the following operations: A nested for-loop where both loops iterate over all the elements passed in as input, a while-loop which iterates over all the elements passed in as input and a simple operation to add two numbers. The complexity in this case will be O(n^2 +n+1).</p>
<p>The complexity is O(n^2) because we need to eliminate the lower order terms and constants. The reason of this is, when stating the complexity of the algorithm, we always assume that the input is a large number of values. When ‘n’ tends to become large, the constants and the terms with lower powers tend to matter less. This is purely by convention and there is no reason we cannot design an algorithm for only a small set of input values, if we are sure about it.</p>
<p>Note that if we have our operation calling another function, then the complexity of the algorithm will need to include the complexity of that algorithm as well.</p>
<p>Here are some Java examples to solidify the concept.</p>
<h2> Big-O Examples</h2>
<h3>Constant</h3>
<p><b>Representation :</b> O(1) <br />
<b>Code :</b> <br /></p>
<figure class="highlight"><pre><code class="language-java" data-lang="java"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="code"><pre><span class="c1">// Checks if the first Character of an array is the character 'A'</span>
<span class="kd">private</span> <span class="kd">static</span> <span class="kt">boolean</span> <span class="nf">isFirstCharA</span><span class="o">(</span><span class="kt">char</span> <span class="o">[]</span> <span class="n">chars</span><span class="o">){</span>
<span class="k">if</span><span class="o">(</span><span class="n">chars</span><span class="o">[</span><span class="mi">0</span><span class="o">]</span> <span class="o">==</span> <span class="sc">'A'</span><span class="o">)</span>
<span class="k">return</span> <span class="kc">true</span><span class="o">;</span>
<span class="k">else</span>
<span class="k">return</span> <span class="kc">false</span><span class="o">;</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p><b>Explanation :</b> <br />
It doesn’t matter what the size of the input array is, the only operation that the function (algorithm) performs is to check if the first element is ‘A’. Hence regardless of the input size, the complexity is O(1).</p>
<h3>Linear</h3>
<p><b>Representation :</b> O(n) <br />
<b>Code :</b> <br /></p>
<figure class="highlight"><pre><code class="language-java" data-lang="java"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
</pre></td><td class="code"><pre><span class="c1">// Finds the largestNumber in the given input integer array</span>
<span class="kd">private</span> <span class="kd">static</span> <span class="kt">int</span> <span class="nf">findLargestNumber</span><span class="o">(</span><span class="kt">int</span><span class="o">[]</span> <span class="n">input</span> <span class="o">)</span> <span class="o">{</span>
<span class="kt">int</span> <span class="n">largestNumber</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span>
<span class="k">for</span><span class="o">(</span><span class="kt">int</span> <span class="nl">current:</span><span class="n">input</span><span class="o">){</span>
<span class="k">if</span><span class="o">(</span><span class="n">current</span> <span class="o">></span> <span class="n">largestNumber</span><span class="o">){</span>
<span class="n">largestNumber</span> <span class="o">=</span> <span class="n">current</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="k">return</span> <span class="n">largestNumber</span><span class="o">;</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p><b>Explanation :</b> <br />
Regardless of where the largest number is in the input integer array, the algorithm needs to traverse all the elements of the array at least once. Hence, considering we have ‘n’ elements, the complexity is O(n).</p>
<h3>Polynomial</h3>
<p><b>Representation :</b> O(n^2) <br />
<b>Code :</b> <br /></p>
<figure class="highlight"><pre><code class="language-java" data-lang="java"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
</pre></td><td class="code"><pre><span class="c1">// Checks if there any number in one array is equal to another </span>
<span class="kd">private</span> <span class="kd">static</span> <span class="kt">boolean</span> <span class="nf">checkIfDuplicatesExist</span><span class="o">(</span><span class="kt">int</span><span class="o">[]</span> <span class="n">input1</span><span class="o">,</span> <span class="kt">int</span><span class="o">[]</span> <span class="n">input2</span><span class="o">)</span> <span class="o">{</span>
<span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">currentInput1</span> <span class="o">:</span> <span class="n">input1</span><span class="o">)</span> <span class="o">{</span>
<span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">currentInput2</span> <span class="o">:</span> <span class="n">input2</span><span class="o">)</span> <span class="o">{</span>
<span class="k">if</span> <span class="o">(</span><span class="n">currentInput1</span> <span class="o">==</span> <span class="n">currentInput2</span><span class="o">)</span> <span class="o">{</span>
<span class="k">return</span> <span class="kc">true</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="k">return</span> <span class="kc">false</span><span class="o">;</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p><b>Explanation :</b> <br />
Notice that there are two for-loops in the above algorithm. O(n^2) complexities appear in cases where there are nested loops. This type of complexity is called polynomial and O(n^2) is specifically known as quadratic complexity. Beginners often come to a conclusion that the number of loops in the algorithm dicates the complexity. This has been true in the case of linear and polynomial complexity, but will be not be true in the case of logarithmic analysis, as we will see next.</p>
<h3>Logarithmic</h3>
<p><b>Representation :</b> O(log n) <br />
<b>Code :</b> <br /></p>
<figure class="highlight"><pre><code class="language-java" data-lang="java"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
</pre></td><td class="code"><pre><span class="c1">// Checks the number of times a given number can be divided by 2 before we end up with a remainder of 1 or less.</span>
<span class="kd">private</span> <span class="kd">static</span> <span class="kt">int</span> <span class="nf">divideByTwo</span><span class="o">(</span><span class="kt">int</span> <span class="n">input</span><span class="o">){</span>
<span class="kt">int</span> <span class="n">counter</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span>
<span class="k">while</span><span class="o">(</span><span class="n">input</span> <span class="o">></span> <span class="mi">1</span><span class="o">){</span>
<span class="n">input</span> <span class="o">=</span> <span class="n">input</span><span class="o">/</span><span class="mi">2</span><span class="o">;</span>
<span class="n">counter</span><span class="o">++;</span>
<span class="o">}</span>
<span class="k">return</span> <span class="n">counter</span><span class="o">;</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p><b>Explanation :</b> <br />
When we give this function an input of ‘64’, the number of times the while loop runs is 6. Coincidentally, the log base 2 of 64 = 6. As we progress through the problem, the number of steps required to reach the end is not linear anymore. It reduces logarithmically. Log n complexities are a good thing.</p>
<h3>Factorial</h3>
<p><b>Representation :</b> O(n!) <br />
<b>Explanation :</b> <br />
A great example for the factorial complexity is the traveling salesman problem. Let’s say there are 3 towns (which are all at different distances from each other) a salesman needs to visit and the problem is to figure out the shortest path to visit all three towns. If we write down the different paths, we will come up with 6 paths, which is nothing but 3! where 3 is the number of towns : <br />
A -> B -> C <br />
A -> C -> B <br />
B -> C -> A <br />
B -> A -> C <br />
C -> A -> B <br />
C -> B -> A <br /></p>
<p>For an input of n, the complexity of such a problem will be O(n!).
Note that half of paths in this case will be of the same distance, just traversed in reverse, so technically the complexity should be O(n!/2). However, remember that we ignore all constants while coming up with the complexity.</p>
<h3>Exponential</h3>
<p><b>Representation :</b> O(2 ^n) <br />
<b>Explanation :</b> <br />
Algorithms where the number of operations required to solve the problem increases exponentially have a complexity of O(2^n). Similar to the factorial complexity, as n tends to be very large, the problems cannot be solved in a reasonable amount of time. For example, a program where n=30 could result in 1 billion operations.</p>
<p>That’s it folks. This post barely skims the surface of all that is possible in algorithm analysis, but I will consider this a success it helps you in the future to do some rough analysis when comparing different approaches to solve a programming problem.</p>
<p>Let me know if I can describe anything better or if you feel like I have misrepresented anything here.
A gist of all the code with some poor man’s tests available <a href="https://gist.github.com/jasti/664bd15745e82ac8a72d">here</a>.</p>
<hr />
Looking For A Career Change? Try Consulting2014-05-24T00:00:00+00:00jasti.github.io/posts/consulting<p>After working for 6 years at Goldman Sachs (GS) doing a mix of software development and project management, I felt like I needed a career change. I was also doing my MBA while working at GS (a post for another time), which encouraged me to look into technology consulting.</p>
<p>I felt that it would allow me to stay technical, solve interesting problems and work with people outside the tech world. After interviewing at a couple of consulting firms, I decided to work in the consulting arm of a financial software company.</p>
<p>A year has passed and having worked on close to 9 different client projects in the past year, I feel like I have a good understanding of the consulting job genre. I didn’t have many people to talk to before considering my current job. I hope that this post will help you if you are in the market looking to do something different.</p>
<p>First, lets get a couple of caveats out of the way :</p>
<p>The consulting I refer to is not the typical Boston Consulting Group or McKinsey strategy type consulting, but specifically software vendor consulting. Although, I hear that there are more similarities than differences between the two jobs, so a lot of what I write here should apply broadly.</p>
<p>I don’t recommend consulting as your first job. The reasons below should give you a better idea why, but it comes down to this one point - learn to crawl before you run.</p>
<p>And finally, the most important of all. I am not referring to the consulting gigs where you are on your own, finding projects by yourself and not working for a software vendor who owns a product. Finding contractor type jobs these days on your own seems quite hard. The competition is cutthroat. Once, when I posted for an hourly consulting job on behalf of a friend on <a href="https://weworkremotely.com/">weworkremotely.com</a>, we received close to a hundred respectable resumes. Not that this type of consulting is bad, but you end up spending a lot of time making yourself marketable. Time that could be spent solving cool problems. If this is your cup of tea, all power to you.</p>
<h2>Being Your Own Boss</h2>
<p>Clients spend quite a bit of money when they hire a consulting firm, so they inherently trust you to know what you are doing. You are not just allowed, but expected to drive the show for things you are responsible for. This has its risks. Anything you do, good or bad has amplified effects. Sometimes this can mean that the clients have very high expectations for the work you deliver. What you get instead is freedom in design choices, control of your work day and power to make things happen instead of helplessly taking orders.</p>
<p>Most consulting firms have a pretty thin management layer. It is because professional services make their revenue by having employees deployed at client sites. This has two great side effects: <br /></p>
<ol>
<li>Your managers are usually also involved in projects at a client, instead of just “managing” you. So they are quite hands-on and can help you solve problems instead of becoming one. <br /></li>
<li>No unnecessary meetings with your managers. <br /></li>
</ol>
<p>The only meetings you attend are the ones <i>you</i> create. You have the freedom to set up a 11 minute meeting (if you want), set a clear agenda, invite only the people you really need for the meeting and assign ownership for delivery. You are in control.</p>
<h2>Professional Development</h2>
<p>Anyone who has worked in a company with more than a 1000 employees knows that the real learning curve takes place during the first year at a job. After that it can be very easy to “glide” through your career unless you take complete charge of it. As a software consultant, you have to be an expert in the area they bring you in for. You need to keep up with the advances in the area and it is an ongoing process.</p>
<p>As a consultant your work is measured by the hour. This is a great way to stay accountable and not fall prey to <a href="http://en.wikipedia.org/wiki/Parkinson's_law"> Parkinson’s Law</a>. You focus more on getting things done vs “finding the best solution” where you could have diminishing returns beyond a certain point.</p>
<p>Your projects only last for about a month or two at a time and you cannot rely on your past accomplishments. You have to gain the client’s trust with every new project. This could be quite de-motivating sometimes but it’s also an opportunity to hit the reset button and learn from your past mistakes. This problem dilutes over time as you spend some time in the industry. Clients that like you want you to come back for other projects.</p>
<p>Consulting firms need to invest quite a bit in you before they put you out in the field because they aim to present a consistent level of expertise. In my case, we had a 4 month training program with topics ranging from finance to the architecture of the product combined with a mock implementation of the product judged by the senior management at the firm. Think about it, since graduation, when was the last time you had the opportunity to really focus for 4 months to learn new skills? And all this while you are getting paid!</p>
<h2>Tighter Feedback </h2>
<p>Since you are so close to the action and you can measure your output day in and day out, you get immediate feedback on your work. As a software consultant, you are brought in to solve very specific problems. You could be migrating a client off another vendor software by building functionality in your own software, or you could be building interfaces for the client to connect to an exotic data provider. You don’t spend months and years building something that the users may or may not like. If the client doesn’t like what they see, you will know very soon. There is no ambiguity in your output and the value you deliver. You are happy when the implementation goes well, or you learn from your mistakes and try to do better the next time.</p>
<h2>Personal Development </h2>
<p>Traveling sucks. After the initial rush of feeling like George Clooney from <a href="https://www.youtube.com/watch?v=rTL1FmvVCuA">Up In The Air</a>, it all goes downhill.
Almost 75% of the flights I have traveled on were delayed. You could spend hours stuck in traffic. Yes, you could get some quality work done on the flights, but this is only effective 50% of the time. When you are going some place from home. The laptop is the last thing you want to look at after a long day with the client. I’m usually brain dead at that point.</p>
<p>During the initial months of consulting, I had to travel every week to Boston for 10 weeks straight and was only spending the weekends at home. You build a special meaning for home and you start to appreciate the little things . Being able to go on a run with your significant other. Being able to have a home-cooked meal. I can’t imagine how much harder this can get if you have kids. All the travel miles, fancy hotels and expensive dinners are not worth any of this.</p>
<p>Thankfully, all my projects after the initial burst have been in and around New York, so it hasn’t been as bad for me. Plus it is much easier to work remotely as a software consultant than any other type of consultant.</p>
<p>There is no sugar-coating it. Traveling sucks. Pick firms that don’t make you travel as much.</p>
<p>With consulting, every day is a new challenge. You walk into work with a fresh set of problems. There are too many variables to predict and prepare. You do your best and come up with a framework to deal with unexpected situations which you internalize over time. This is exactly the kind of thing I never learned from the days at GS working exclusively as a software developer. As a software developer, it is very easy to get reclusive and stay in your computer cave. With software consulting, you obviously have to be good at software development, but aside from that you learn skills like conflict resolution, negotiation, writing and speaking. As a result you push yourself to be a more well-rounded person.</p>
<p>You also meet people who are very different than you. It becomes a natural networking opportunity. ‘C’ level executives, sales people who sold your software product to the client, traders, portfolio managers, tech people at the client firm and other software vendors like Bloomberg or MarkIt. This is a great way to learn cross-functionally. You get to see other job functions and compare your job to theirs. It’s always interesting to know the small details of what other people do in their day job.</p>
<p>You are pushed to become more emotionally-balanced and professionally mature as a result of working with so many people. Over time you start to become more thick-skinned. This could be a good thing. As a society, we worry too much about what other people think of us anyway. You’ll be more sure about yourself and be open for constructive criticism because you know to weed the good feedback from the bad.</p>
<h2>Doing What You Love</h2>
<p>Consulting is not for everyone, and it is especially not for you if you already love what you do. I found software consulting to be refreshingly interesting after working for a thirty-five thousand person company. Your experience may be different, and you can find exceptions to every point I mention here and I respect that. I encourage you to get in touch with me if you feel differently. A role of a software consultant is just not as concretely defined as, say, a web user interface developer.</p>
<p>If you are thinking of making a career switch, I hope this post will give you a slight nudge to consider consulting as a career option, if you haven’t already.</p>
<hr />
A Programmer's Recipe For Scraping The Web2014-05-20T00:00:00+00:00jasti.github.io/posts/scraping<p>There is a lot of data locked behind the websites we visit every day. What you see on a page is usually just a tiny sliver of all the information that a website hosts. Some websites expose all this data via an <a href="http://en.wikipedia.org/wiki/Application_programming_interface">API</a>, but most don’t. <br /></p>
<p>What if you want data from websites that don’t provide an API? A good but inefficient way is to open different links of the website, copy the data you want, manually, into a format that you can later use for data analysis. Is there a better way to get hold of all this data without spending a ridiculous amount of time engaged in mind numbing tasks? Yes, there is. In fact, Google indexing works using a similar paradigm. This is exactly the type of task computers are good at.</p>
<p>Having done this a couple of times, I want to share a framework to deal with such problems using an example.</p>
<h2>Define The Problem</h2>
<p>This sounds very obvious, but your ability to get the right data heavily depends on this step. Write it down.<br />
Let us pick an example in our case. I enjoy Crossfit. It has a lot of positives with a few downsides (a topic for another day) and I follow the daily workouts posted on <a href="http://crossfit.com/">crossfit.com</a>. This is a website maintained by the official headquarters and they have been posting a workout a day since the year 2002. My goal is to get all the workouts posted since 2002 in the format below into a tab separated file:</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">Date</span> <span class="n">Workout</span>
<span class="mi">20140519</span> <span class="n">Complete</span> <span class="k">as</span> <span class="n">many</span> <span class="n">rounds</span> <span class="k">as</span> <span class="n">possible</span> <span class="ow">in</span> <span class="mi">10</span> <span class="n">minutes</span> <span class="n">of</span><span class="p">:</span> <span class="mi">5</span> <span class="n">bar</span> <span class="n">muscle</span><span class="o">-</span><span class="n">ups</span> <span class="mi">135</span><span class="o">-</span><span class="n">lb</span><span class="p">.</span> <span class="n">thrusters</span><span class="p">,</span> <span class="mi">10</span> <span class="n">reps</span></code></pre></figure>
<h2>Pick Your Tools</h2>
<p><i> The post assumes that you know how to install the necessary tools on your machine. I will provide the necessary links to where you can find the software, but showing you how to use <a href="http://pip.readthedocs.org/en/latest/installing.html">pip</a> gets boring very quickly :)</i></p>
<p>There are many tools available for scraping the web but I urge you to pick tools that give you the most amount of flexibility. I automatically lean towards programming instead of out of the box tools for obvious advantages like customizability, accommodating requirement changes and reusability. Python is great at these tasks because it is, what I call, a glue language. Python has had continued interest in different domains and that allows Python interface easily between data munging, scraping, transformation, visualization and hosting web applications.</p>
<p>We’ll do this example in Python and use a excellent library called <a href="http://www.crummy.com/software/BeautifulSoup/"> Beautiful Soup</a>.</p>
<p>For the IDE,<ahref =”http://ipython.org/notebook.html”> IPython</a> is fantastic. The advantage of IPython is the ability to re-run segments of your code instead of the entire program, which is excellent when you are experimenting with unknown data or figuring out language capabilities. The last library we need is<ahref =”http://pandas.pydata.org/”> Pandas</a>. We technically don’t <i>need </i> Pandas, but it makes it easier by allowing us to transform data with the help of multi-dimensional array manipulations. It is used a lot in the scientific community and is a wrapper on top of Numpy and Scipy. If you haven’t dealt with Pandas before, I recommend you look at <a href="http://vimeo.com/59324550">this</a> quick intro from the creator, Wes McKinney or I highly recommend his <a href="http://www.amazon.com/Python-Data-Analysis-Wrangling-IPython/dp/1449319793">book</a>.</p>
<p>If you are on a Mac, you can use Chris Fonnesbeck’s <a href="https://github.com/fonnesbeck/ScipySuperpack">install script</a> that will download Pandas and all its dependencies. If you are on Windows, you can use the free version of the<a href="https://www.enthought.com/products/epd/free/"> Enthought distribution</a> install. It does a pretty good job.</p>
<p>I am personally not a big fan of the debugging capabilities that come with <a href="https://docs.python.org/2/library/pdb.html">Python</a> and I almost always rely on an IDE like <a href="http://pydev.org/">PyDev</a> or <a href="http://spyder-ide.blogspot.com/">Spyder</a> when I want to debug some code. My workflow ends up going back and forth between IPython for experimentation and PyDev for debugging. Long story short, pick tools that you are comfortable with and be flexible to switch between tools.</p>
<h2>Discover Patterns</h2>
<p>Now that we are all set up, on to the good stuff! <br />
Every web page on a website has a blueprint. Think of each blueprint made of boxes of text and each of these boxes could contain one or more boxes. Our goal is to create a hierarchy of boxes we care about, programmatically using Python and Beautiful Soup. <br />
Navigate to a site like <a href="http://crossfit.com/">Crossfit.com</a>. Right click on the text that we are interested in within Chrome and select inspect element. This will display the HTML rendered by the site in the developer tools, like below.</p>
<center><img src="/images/blog/scraping/inspect.png" /></center>
<p><br />
The “boxes” we care about in this document are : <br /></p>
<ul>
<li>Everything within the ‘div’ element where the class is ‘blogbody’<br />
++ The ‘h3’ element where the class name is ‘title’. This is the date the workout was posted on. <br />
++ The ‘p’ element which is the workout description.<br /></li>
</ul>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
</pre></td><td class="code"><pre><span class="kn">from</span> <span class="nn">datetime</span> <span class="kn">import</span> <span class="n">datetime</span>
<span class="kn">import</span> <span class="nn">calendar</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="n">np</span>
<span class="kn">from</span> <span class="nn">bs4</span> <span class="kn">import</span> <span class="n">BeautifulSoup</span>
<span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">import</span> <span class="nn">csv</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s">"workouts.tsv"</span><span class="p">,</span> <span class="s">"w+"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">fieldnames</span> <span class="o">=</span> <span class="p">(</span><span class="s">"Date"</span><span class="p">,</span> <span class="s">"Workout"</span><span class="p">)</span>
<span class="n">output</span> <span class="o">=</span> <span class="n">csv</span><span class="p">.</span><span class="n">writer</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="n">delimiter</span><span class="o">=</span><span class="s">"</span><span class="se">\t</span><span class="s">"</span><span class="p">)</span>
<span class="n">output</span><span class="p">.</span><span class="n">writerow</span><span class="p">(</span><span class="n">fieldnames</span><span class="p">)</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">"http://www.crossfit.com/"</span>
<span class="n">r</span> <span class="o">=</span> <span class="n">requests</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="n">soup</span> <span class="o">=</span> <span class="n">BeautifulSoup</span><span class="p">(</span><span class="n">r</span><span class="p">.</span><span class="n">text</span><span class="p">)</span>
<span class="c1">#print soup
</span> <span class="n">blogBodies</span> <span class="o">=</span> <span class="n">soup</span><span class="p">.</span><span class="n">findAll</span><span class="p">(</span><span class="s">"div"</span><span class="p">,</span> <span class="p">{</span><span class="s">"class"</span> <span class="p">:</span> <span class="s">"blogbody"</span><span class="p">})</span>
<span class="k">for</span> <span class="n">line</span> <span class="ow">in</span> <span class="n">blogBodies</span><span class="p">:</span>
<span class="c1">#print('\n~break~\n');
</span> <span class="n">workoutName</span> <span class="o">=</span> <span class="n">line</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">"h3"</span><span class="p">,</span> <span class="p">{</span><span class="s">"class"</span> <span class="p">:</span> <span class="s">"title"</span><span class="p">})</span>
<span class="c1">#print workoutName
</span> <span class="k">if</span> <span class="n">workoutName</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span><span class="p">:</span>
<span class="k">if</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">workoutName</span><span class="p">.</span><span class="n">text</span><span class="p">.</span><span class="n">split</span><span class="p">())</span> <span class="o"><</span> <span class="mi">2</span><span class="p">):</span>
<span class="c1"># Some entries only have the day and not the actual date
</span> <span class="c1">#workoutDate = np.nan;
</span> <span class="k">break</span><span class="p">;</span>
<span class="k">else</span><span class="p">:</span>
<span class="c1">#Add 20 to the year part, so Pandas can easily extract the date format
</span> <span class="n">workoutDate</span> <span class="o">=</span> <span class="s">"20"</span><span class="o">+</span><span class="n">workoutName</span><span class="p">.</span><span class="n">text</span><span class="p">.</span><span class="n">split</span><span class="p">()[</span><span class="mi">1</span><span class="p">];</span>
<span class="n">para</span> <span class="o">=</span><span class="n">line</span><span class="p">.</span><span class="n">findAll</span><span class="p">(</span><span class="s">'p'</span><span class="p">)</span>
<span class="n">workout</span> <span class="o">=</span> <span class="s">""</span><span class="p">;</span>
<span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">para</span><span class="p">:</span>
<span class="c1"># Sometime their workouts have links and they need to be filtered for
</span> <span class="n">anchorExists</span> <span class="o">=</span> <span class="n">p</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">"a"</span><span class="p">)</span>
<span class="k">if</span> <span class="p">(</span><span class="n">p</span><span class="p">.</span><span class="n">text</span><span class="o">==</span><span class="s">'Enlarge image'</span><span class="p">)</span> <span class="o">|</span><span class="p">(</span><span class="s">'Post'</span> <span class="ow">in</span> <span class="n">p</span><span class="p">.</span><span class="n">text</span><span class="p">)</span><span class="o">|</span><span class="p">(</span><span class="s">'comments'</span> <span class="ow">in</span> <span class="n">p</span><span class="p">.</span><span class="n">text</span><span class="p">)</span><span class="o">|</span><span class="p">(</span><span class="n">p</span><span class="p">.</span><span class="n">text</span> <span class="o">==</span> <span class="s">""</span><span class="p">)</span> <span class="p">:</span>
<span class="k">break</span><span class="p">;</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">workout</span> <span class="o">=</span> <span class="n">workout</span><span class="o">+</span><span class="n">p</span><span class="p">.</span><span class="n">text</span><span class="p">.</span><span class="n">encode</span><span class="p">(</span><span class="s">'utf-8'</span><span class="p">)</span>
<span class="n">output</span><span class="p">.</span><span class="n">writerow</span><span class="p">([</span><span class="n">workoutDate</span><span class="p">,</span><span class="n">workout</span><span class="p">.</span><span class="n">strip</span><span class="p">().</span><span class="n">replace</span><span class="p">(</span><span class="s">'</span><span class="se">\n</span><span class="s">'</span><span class="p">,</span><span class="s">' '</span><span class="p">)])</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>OK, like with any piece of code, there seems to be a lot going on here. I promise, it’s not the case. Take a deep breath and let’s break it down :</p>
<p>line 8 : Open a file where we want to write our output <br />
line 12: The URL we want to parse <br />
line 14: Convert our site to a Beautiful Soup template that we can easily parse and query<br />
line 16: Grab all the ‘div’ elements that have the class name as ‘blogbody’. (Remember, this was our main container)<br />
line 19: Grab all the ‘h3’ elements that have the class name as ‘title’. (This is the workout date)<br />
line 29: Grab all the ‘p’ elements from each line in ‘blogbodies’. (This is the workout name and description)<br /></p>
<p>Ignore the other lines for now. We will tackle them in the “Dealing With Gotchas” section below, which are just special cases to get around data quirks.</p>
<p>Your output will look like this :</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
</pre></td><td class="code"><pre><span class="n">Date</span> <span class="n">Workout</span>
<span class="mi">20140519</span> <span class="n">Complete</span> <span class="k">as</span> <span class="n">many</span> <span class="n">rounds</span> <span class="k">as</span> <span class="n">possible</span> <span class="ow">in</span> <span class="mi">10</span> <span class="n">minutes</span> <span class="n">of</span><span class="p">:</span> <span class="mi">5</span> <span class="n">bar</span> <span class="n">muscle</span><span class="o">-</span><span class="n">ups</span> <span class="mi">135</span><span class="o">-</span><span class="n">lb</span><span class="p">.</span> <span class="n">thrusters</span><span class="p">,</span> <span class="mi">10</span> <span class="n">reps</span>
<span class="mi">20140518</span> <span class="n">Rest</span> <span class="n">Day</span>
<span class="mi">20140517</span> <span class="n">Run</span> <span class="mi">1600</span> <span class="n">meters</span> <span class="n">Rest</span> <span class="mi">3</span> <span class="n">minutes</span> <span class="n">Run</span> <span class="mi">1200</span> <span class="n">meters</span> <span class="n">Rest</span> <span class="mi">2</span> <span class="n">minutes</span> <span class="n">Run</span> <span class="mi">800</span> <span class="n">meters</span> <span class="n">Rest</span> <span class="mi">1</span> <span class="n">minute</span> <span class="n">Run</span> <span class="mi">400</span> <span class="n">metersCompare</span> <span class="n">to</span> <span class="mf">121017.</span>
<span class="mi">20140516</span> <span class="mi">50</span><span class="o">-</span><span class="mi">40</span><span class="o">-</span><span class="mi">30</span><span class="o">-</span><span class="mi">20</span><span class="o">-</span><span class="mi">10</span> <span class="n">reps</span> <span class="k">for</span> <span class="n">time</span> <span class="n">of</span><span class="p">:</span> <span class="n">GHD</span> <span class="n">sit</span><span class="o">-</span><span class="n">ups</span> <span class="mi">2</span><span class="o">-</span><span class="n">pood</span> <span class="n">kettlebell</span> <span class="n">swings</span>
<span class="mi">20140515</span> <span class="mi">7</span> <span class="n">rounds</span> <span class="n">of</span><span class="p">:</span> <span class="mi">15</span> <span class="n">hip</span><span class="o">-</span><span class="n">back</span> <span class="n">extensions</span> <span class="mi">3</span> <span class="n">strict</span> <span class="n">muscle</span><span class="o">-</span><span class="n">ups</span>
<span class="mi">20140514</span> <span class="n">Rest</span> <span class="n">Day</span>
<span class="mi">20140513</span> <span class="n">Helton3</span> <span class="n">rounds</span> <span class="k">for</span> <span class="n">time</span> <span class="n">of</span><span class="p">:</span> <span class="n">Run</span> <span class="mi">800</span> <span class="n">meters</span> <span class="mi">50</span><span class="o">-</span><span class="n">lb</span><span class="p">.</span> <span class="n">dumbbell</span> <span class="n">squat</span> <span class="n">clean</span><span class="p">,</span> <span class="mi">30</span> <span class="n">reps</span> <span class="mi">30</span> <span class="n">burpees</span>
<span class="mi">20140512</span> <span class="n">Front</span> <span class="n">squat</span> <span class="mi">3</span><span class="o">-</span><span class="mi">3</span><span class="o">-</span><span class="mi">3</span><span class="o">-</span><span class="mi">3</span><span class="o">-</span><span class="mi">3</span> <span class="n">reps</span>
<span class="mi">20140511</span> <span class="n">Complete</span> <span class="k">as</span> <span class="n">many</span> <span class="n">rounds</span> <span class="k">as</span> <span class="n">possible</span> <span class="ow">in</span> <span class="mi">20</span> <span class="n">minutes</span> <span class="n">of</span><span class="p">:</span> <span class="n">Run</span> <span class="mi">400</span> <span class="n">meters</span> <span class="n">Then</span> <span class="mi">3</span> <span class="n">rounds</span> <span class="n">of</span><span class="p">:</span> <span class="mi">5</span> <span class="n">pull</span><span class="o">-</span><span class="n">ups</span> <span class="mi">10</span> <span class="n">push</span><span class="o">-</span><span class="n">ups</span> <span class="mi">15</span> <span class="n">squatsCompare</span> <span class="n">to</span> <span class="mf">121119.</span>
<span class="mi">20140510</span> <span class="n">Rest</span> <span class="n">Day</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>The date of the workout followed by the workout description.<br />
That’s it! The code above constructs the blueprint and we just need to repeat this process for all the URLs that we need data from, which we will look at in the next section.</p>
<h2>Construct URLs Recursively</h2>
<p>In our case, if we navigate to a URL like “http://www.crossfit.com/mt-archive2/YYYY_MM.html”, we will have access to all the archives. Again, it’s just about finding the pattern, like in the previous section.
Once we have the pattern, write a simple method to auto-generate our URLs :</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
</pre></td><td class="code"><pre><span class="k">def</span> <span class="nf">generateURL</span><span class="p">():</span>
<span class="n">crossfitBaseUrl</span> <span class="o">=</span> <span class="s">'http://www.crossfit.com/mt-archive2/YYYY_MM.html'</span>
<span class="n">yearRange</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">2002</span><span class="p">,</span> <span class="mi">2014</span><span class="p">)</span>
<span class="n">monthRange</span> <span class="o">=</span> <span class="p">[</span><span class="s">'01'</span><span class="p">,</span><span class="s">'02'</span><span class="p">,</span><span class="s">'03'</span><span class="p">,</span><span class="s">'04'</span><span class="p">,</span><span class="s">'05'</span><span class="p">,</span><span class="s">'06'</span><span class="p">,</span><span class="s">'07'</span><span class="p">,</span><span class="s">'08'</span><span class="p">,</span><span class="s">'09'</span><span class="p">,</span><span class="s">'10'</span><span class="p">,</span><span class="s">'11'</span><span class="p">,</span><span class="s">'12'</span><span class="p">]</span>
<span class="n">crossfitUrls</span> <span class="o">=</span> <span class="p">[];</span>
<span class="k">for</span> <span class="n">year</span> <span class="ow">in</span> <span class="n">yearRange</span><span class="p">:</span>
<span class="k">for</span> <span class="n">month</span> <span class="ow">in</span> <span class="n">monthRange</span><span class="p">:</span>
<span class="n">crossfitYearUrl</span> <span class="o">=</span> <span class="n">crossfitBaseUrl</span><span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">'YYYY'</span><span class="p">,</span> <span class="nb">str</span><span class="p">(</span><span class="n">year</span><span class="p">))</span>
<span class="n">crossfitUrl</span> <span class="o">=</span> <span class="n">crossfitYearUrl</span><span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">'MM'</span><span class="p">,</span> <span class="nb">str</span><span class="p">(</span><span class="n">month</span><span class="p">))</span>
<span class="n">crossfitUrls</span><span class="p">.</span><span class="n">append</span><span class="p">(</span><span class="n">crossfitUrl</span><span class="p">);</span>
<span class="k">return</span> <span class="n">crossfitUrls</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Iteratively pass in the output from the generateURL() method to the code in the previous section and that will print workout dates and workout names to the tab separated file.</p>
<h2>Munge Your Data</h2>
<p>Once our data is written to the tab separated file, we can ‘clean’ rows, deal with date formats, check if we have all the data, the right data, transform or munge data as necessary. <br />
I realize that this post has grown much larger than I intended it to be, so we’ll look at the transformation of this data using Pandas as part of another post.<br />
For now, below is a simple way to move all our data to a DataFrame. Google up for some DataFrame tutorials and try applying some transformations. Once you get comfortable dealing with multi-dimensional arrays (which is essentially what a DataFrame is) you will find Pandas to be efficient and enjoyable.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="code"><pre><span class="n">cols</span> <span class="o">=</span><span class="p">[</span><span class="s">'Date'</span><span class="p">,</span><span class="s">'Workout'</span><span class="p">]</span>
<span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="p">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s">'workouts.tsv'</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s">'</span><span class="se">\t</span><span class="s">'</span><span class="p">,</span> <span class="n">converters</span><span class="o">=</span><span class="p">{</span><span class="s">'Date'</span><span class="p">:</span> <span class="nb">str</span><span class="p">})</span>
<span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">.</span><span class="n">dropna</span><span class="p">()</span>
<span class="k">print</span> <span class="n">df</span>
</pre></td></tr></tbody></table></code></pre></figure>
<h2>Dealing With Gotchas</h2>
<p>On to the lines we skipped above. As we encounter more example of data parsing, your biggest time sucker is programming <a href="http://en.wikipedia.org/wiki/Defensive_programming">defensively</a>. You start to encounter so many unforeseen circumstances that it’s hard to code against every single case.</p>
<p>line 22: Sometimes, the ‘workoutName’ has just the weekday and doesn’t have the actual date the workout was posted<br />
line 28: Had to manually add the “20” part of the year for Python to recognize the date as an actual date format<br />
line 34: Not all ‘<p>’s consist of the workout description, some of them could be comments or embedded images. Just a simple way to ignore such texts<br />
line 38: Avoid new lines and remove empty spaces from workouts <br /></p>
<h2>Some Broader Ideas</h2>
<p>Following a couple of rules has helped me parse web sites more efficiently. In fact, we can apply these principles if we are hacking on any unstructured code or tiny projects:</p>
<h3>Iterative Development</h3>
<p>Build it - build it right - simplify it - and optionally, make it fast.
Don’t worry about getting all the cases right in the first go about. Just get something down and refine to get the right data. Once you have it, simplify your code to make it modular and reusable. As we did above, start with a single URL, then try parsing the data for a month. You will encounter some unforeseen issues, fix them and try parsing the data for a couple of months and so on..</p>
<h3>Reusability</h3>
<p>You will deal with the same sought of problems like parsing dates, escaping unicode characters, generating URLs, reading from files, writing to files. Create a Python module and reuse the methods. The best code is the code you don’t write at all.
I plan to open-source my reusable module for web parsing sometime soon after some tidying up. There is just too much stuff in one big module at this point.</p>
<h3>Print Statements For Debugging</h3>
<p>I don’t care what the haters say, I have found tremendous use of just printing output instead of launching the full on debugger but you should definitely look into learning a tool like Pydev( a Python plugin for Eclipse) or Spyder for some variable inspection level debugging.</p>
<p>Good luck and let me know if you have any questions or if I can improve this post in anyway. Happy hacking!</p>
<p>As always, you can find the source for this on my <a href="https://github.com/jasti/CrossfitScraper">Github account</a>.
You can also find an IPython notebook with the code <a href="https://github.com/jasti/CrossfitScraper/blob/master/NBCrossfitScraper.ipynb">here</a>.</p>
<hr />
Build Your Own Blog2014-05-08T00:00:00+00:00jasti.github.io/posts/blog<p>Writing here has gotten me thinking more deeply about things I come across everyday. Talking comes naturally to me and I can spend days having conversations with friends with no real agenda, but I am not as good at writing concrete material and I wanted to work towards changing that. Thinking while speaking with someone usually gets me to a level or two, but writing down forces me to solidify my reflection on a thought.</p>
<p>It’s gotten me to re-read what I create multiple number of times, which is a great patience builder. Plus it’s always a good feeling to hear back from that one angel of a reader who emails you to tell you how she found your technical article helpful.</p>
<p>Blog sites like Medium and Tumblr have come a long way with <a href="http://en.wikipedia.org/wiki/WYSIWYG">WYSIWYG</a> but they all claim to do whatever they want with your content, including monetize. Not to say that I won’t write for free, I will, but at my terms.</p>
<p>Excerpt from <a href="https://medium.com/p/9db0094a1e0f">Medium’s terms of service</a></p>
<blockquote>
By posting or transferring content to Medium, you give us permission to use your content solely to do the things we need to do to provide Medium Services, including, without limitation, storing, displaying, reproducing, and distributing your content. This may include promoting your content with partner companies or services for broader broadcast, distribution, or publication.
</blockquote>
<p><br /></p>
<p>There are some other obvious reasons to avoid blogging sites. If they <a href="http://en.wikipedia.org/wiki/Posterous#Shutting_down_Posterous_and_building_PostHaven">shut down</a> they take all your permalinks with them. Switching costs of blog sites are quite high because they make it deliberately hard to export content. Most of them need quite a bit of tweaking to display selectable code snippets or things like quotes.</p>
<p>I started using <a href="http://www.sublimetext.com/">Sublime Text</a> for programming about a year ago and it has been a total game-changer. If you are a programmer, you’ll feel right at home, like you are typing on the command line. Especially if you use the <a href="https://github.com/buymeasoda/soda-theme/">Soda-theme</a>. I like the ability to do most of the things I want to from the command line or Sublime without having to deal with clunky pixelated user interfaces. The ability to write a blog like we build software - iteratively, with source control and experimentation was a huge requirement.</p>
<p>When I decided to write, I started to research and the obvious pick was to use a <a href="http://staticsitegenerators.net/">Static Site Generator</a> and since I was building a rails site for a friend’s startup, I decided to go with <a href="http://jekyllrb.com/">Jekyll </a>. After hacking on it for a couple of weekends, I finally have something that I am quite happy with.
Ultimately, the ability to experiment, learn and see your new features come to fruition instantly, feels quite rewarding.</p>
<p><img src="/images/blog/sublime.png" /></p>
<h2>Jekyll</h2>
<p>I started with Zach Holman’s <a href="https://github.com/holman/left">left theme</a> to work off of a seasoned Jekyll theme. There is no point re-inventing the wheel for this. Yoda once said (or did he?) “If you don’t know your destination, it’s best to walk sideways or backwards than walking forward.” I ended tweaking it to my liking and the biggest change you might notice is in the way the site is laid out and how it looks on mobile.
Using Jekyll’s lightly document ‘watch’ commmand, you can make edits in your editor and have jekyll reload:</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c1"># '-watch' instead of '-w' woudn't autogenerate files for some reason
</span><span class="n">jekyll</span> <span class="n">serve</span> <span class="o">-</span><span class="n">w</span></code></pre></figure>
<h2>GitHub Pages</h2>
<p>Hosting a site of your own has never been easier. All you need is a Github account and a <a href="https://github.com/jasti/jasti.github.io"> repository</a> to publish your static site. It becomes easier if you plan to use Jekyll because Github will do the site generation bit for you directly serve up the generated HTML and JS.
By default, the site is available, in my case, at jasti.github.com which gets forwarded from <a href="http://vamseejasti.com">vamseejasti.com</a>. To make this happen, it just takes one line in the <a href="http://en.wikipedia.org/wiki/CNAME_record">CNAME</a> file that goes with your static site and Github will replace *.github.io with your custom domain name.
If you plan to use a domain manager to buy a site, I recommend <a href="https://iwantmyname.com/">iwantmyname.com</a> and has a one click DNS forwarding with Github.</p>
<h2>Design</h2>
<p>Icons are built with <a href="http://fortawesome.github.io/Font-Awesome/">Font Awesome</a>. The icons are all vector graphics, which look good on any screen resolution and a lot of the compatibility issues are handled for you.
I use the <a href="https://www.google.com/fonts/specimen/Oxygen">Oxygen</a> font for all the content and <a href="http://www.google.com/fonts/specimen/Lobster">Lobster</a> for a header from Google fonts.</p>
<p>This is probably just me but I feel anxious when there are a ton of links on a page I want to read. It feels like an unattended to-do list that can be checked off by clicking on the links. Naturally I want to avoid them as much as possible, but I also want the ability to give the user a deeper experience <i>if they choose to</i>. Played around with some CSS and ended up with all links in a bolded format which I think is less distracting instead of the default underlined blue link.</p>
<p>I use <a href="http://pygments.org/">Pygments</a> for syntax highlighting. It’s simple and does the job. Example below:</p>
<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"><span class="kd">var</span> <span class="nx">http</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">http</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">fs</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">fs</span><span class="dl">'</span><span class="p">);</span>
<span class="nx">fs</span><span class="p">.</span><span class="nx">readFile</span><span class="p">(</span><span class="dl">'</span><span class="s1">index.html</span><span class="dl">'</span><span class="p">,</span> <span class="kd">function</span> <span class="p">(</span><span class="nx">err</span><span class="p">,</span> <span class="nx">html</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// Your html test file</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">err</span><span class="p">)</span> <span class="p">{</span>
<span class="k">throw</span> <span class="nx">err</span><span class="p">;</span>
<span class="p">}</span>
<span class="nx">http</span><span class="p">.</span><span class="nx">createServer</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">request</span><span class="p">,</span> <span class="nx">response</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">response</span><span class="p">.</span><span class="nx">writeHeader</span><span class="p">(</span><span class="mi">200</span><span class="p">,</span> <span class="p">{</span><span class="dl">"</span><span class="s2">Content-Type</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">text/html</span><span class="dl">"</span><span class="p">});</span> <span class="o">\</span>
<span class="nx">response</span><span class="p">.</span><span class="nx">write</span><span class="p">(</span><span class="nx">html</span><span class="p">);</span>
<span class="nx">response</span><span class="p">.</span><span class="nx">end</span><span class="p">();</span>
<span class="p">}).</span><span class="nx">listen</span><span class="p">(</span><span class="mi">8124</span><span class="p">,</span> <span class="dl">'</span><span class="s1">127.0.0.1</span><span class="dl">'</span><span class="p">);</span>
<span class="p">});</span></code></pre></figure>
<h2>Responsive</h2>
<p>The site is responsive and renders appropriately for screens of smaller resolutions by just loading the mobile.css instead of base.css from the css folder. No magic there.</p>
<center>
<img src="/images/blog/mobile.png" />
</center>
<h2>Google Analytics</h2>
<p>I am a junkie to know how often the blog is being visited and by whom. Google Analytics makes this really easy. Just sign up, link your site and insert the snippet of code in the pages you want to track and you are all set. Easy peasy.</p>
<h2>Disqus</h2>
<p>Today was the first time I tried out <a href="http://disqus.com/">Disqus</a> and it was a pleasant surprise. Sign up, generate your custom code and paste your custom code in the footer of your site. The neat thing is that you can carry your Disqus comments to another site if you like, with just a couple of line changes.</p>
<p><img src="/images/blog/disqus.png" /></p>
<h2>Open Source</h2>
<p>I have stripped out my site and created an open source template of the site. So fork it, delete what you don’t need and tweak it to your liking.</p>
<p>You’ll find the source on Github : <a href="https://github.com/jasti/stout">Stout</a></p>
<p>If you plan to use it, I’d be happy if you gave me a shout on <a href="https://twitter.com/vamseejasti" target="_blank"><i class="icon icon-twitter"></i></a>. I’d still be happy if you didn’t tell me, but whatever you do, do not forget the second rule of Fight Club.</p>
<blockquote>
I've already got the prize. The prize is the pleasure of finding the thing out, the kick in the discovery, the observation that other people use it. Those are the real things. - <a href="https://www.youtube.com/watch?v=Bgaw9qe7DEE#t=1466">Richard Feynman</a>
</blockquote>
<hr />
Does Your API Empathize With Its User?2014-04-27T00:00:00+00:00jasti.github.io/posts/empathy<p>I am appalled at the difference in API usability between an API developed for B2B (Business to Business) vs. one developed for B2C (Business to Customer). A good example of a B2B API in the finance world is the <a href="http://www.swift.com/products_services/by_type/messaging/index">15022 swift messaging</a> (primarily used for communication between two financial institutions) which look like a 5 year old’s half eaten bowl of spaghetti. Compare that to a B2C API like <a href="https://parse.com/docs/rest">Parse</a> (a Backend as a service application) which has a beautifully designed API which is a pleasure to work with as a developer.</p>
<p>The biggest reason for this huge disparity? I call it developer empathy.</p>
<p>Parse’s API developer is able to empathize with her customer base and ask an important question - “What would I like to see in this API?”. This is primarily because most technology companies build products that the software engineers can inherently understand. Contrast that to a finance company. A business analyst is usually involved who “translates requirements” for the developer. The developer can’t relate to the product anymore. The task becomes a chore for the developer and chances are that he won’t be as motivated to build a kick-ass product. Not to mention the things that get lost in translation when you start involving a business analyst.</p>
<p>As an aside, I strongly feel that everybody who considers themselves part of the tech department must be able to program. Programming languages are so easy nowadays and there are so many resources online that anyone should be able to at least checkout, change some lines of code and commit to staging. Heck, maybe even prod. This reduces the back and forth cycle of “Can you please restrict this report to weekdays?” type of questions by at least half. Allows both developer and BA to be on the same page and allows tighter iterations of the product. Anyway, I digress.</p>
<p>Consumer-facing tech companies have begun to <a href="http://en.wikipedia.org/wiki/Eating_your_own_dog_food">dog food</a> two types of products: <br /></p>
<ol>
<li>Building products which the software engineers themselves could be users of. E.g. <a href="https://www.gmail.com">Gmail</a> <br /></li>
<li>Hardware and internal tools. E.g <a href="https://cloud.google.com/ ">Google Cloud </a> <br /></li>
</ol>
<p>The second type of dogfooding is interesting. Making products like Google Cloud and Amazon AWS available for public developers / customers, the companies are inherently making these businesses client facing / sexy. They turn into revenue generating businesses. As an added bonus, this gives the company a QA base that is as big as their customer base. Internal software developers empathize with the products they build.</p>
<p>Products obviously need to get built even if they are not customer facing and companies need to start thinking about introducing developer empathy in the B2B world. The solution is to simplify standards and treat the end consumers of the businesses not as businesses, but as individual developers.</p>
<hr />
Webservers That Serve You2014-04-23T00:00:00+00:00jasti.github.io/posts/webserver<p>I feel like I had been living under a rock for all my web development years. Did you know that the default Python installation comes built in with a Http server?!</p>
<p>Before I get hit by a bus, I want to share this with you :</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">python</span> <span class="o">-</span><span class="n">m</span> <span class="n">SimpleHTTPServer</span></code></pre></figure>
<p>Run this anywhere on the command line( ofcourse, make sure Python is installed and is part of the PATH variable)
And that’s it, you are good to go my friend. Launch a browser window and navigate to localhost:8000/</p>
<p>Cheers to being able to quickly test dynamic HTML/JavaScript changes in the browser without the need for setting up
Tomcat, which by the way is a giant pain in the rear to maintain, when compared to this gem.</p>
<p>You can accomplish something similar with node, if you are looking for a really simple webserver.</p>
<p>Assuming you have node installed, start by creating a file called nodeserver.js and copy the contents below into it.</p>
<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"><span class="kd">var</span> <span class="nx">http</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">http</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">fs</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">fs</span><span class="dl">'</span><span class="p">);</span>
<span class="nx">fs</span><span class="p">.</span><span class="nx">readFile</span><span class="p">(</span><span class="dl">'</span><span class="s1">index.html</span><span class="dl">'</span><span class="p">,</span> <span class="kd">function</span> <span class="p">(</span><span class="nx">err</span><span class="p">,</span> <span class="nx">html</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// Your html test file</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">err</span><span class="p">)</span> <span class="p">{</span>
<span class="k">throw</span> <span class="nx">err</span><span class="p">;</span>
<span class="p">}</span>
<span class="nx">http</span><span class="p">.</span><span class="nx">createServer</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">request</span><span class="p">,</span> <span class="nx">response</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">response</span><span class="p">.</span><span class="nx">writeHeader</span><span class="p">(</span><span class="mi">200</span><span class="p">,</span> <span class="p">{</span><span class="dl">"</span><span class="s2">Content-Type</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">text/html</span><span class="dl">"</span><span class="p">});</span> <span class="o">\</span>
<span class="nx">response</span><span class="p">.</span><span class="nx">write</span><span class="p">(</span><span class="nx">html</span><span class="p">);</span>
<span class="nx">response</span><span class="p">.</span><span class="nx">end</span><span class="p">();</span>
<span class="p">}).</span><span class="nx">listen</span><span class="p">(</span><span class="mi">8124</span><span class="p">,</span> <span class="dl">'</span><span class="s1">127.0.0.1</span><span class="dl">'</span><span class="p">);</span>
<span class="p">});</span></code></pre></figure>
<p>Save the file and run</p>
<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"><span class="nx">node</span> <span class="nx">nodeserver</span><span class="p">.</span><span class="nx">js</span></code></pre></figure>
<p>And just like that, you have your own little test webserver by navigating to localhost:8124/ . Create a file called index.html in the same folder and go nuts with linking other HTML/JavaScript files in there.
I love the simplicity of node in creating this webserver because almost everything you would usually configure in a webserver is right here in front of your eyes. Having said that, this is obviously an abstraction and there is a ton of other things going on in the ‘createServer’ statement which I don’t get, but that is a topic for another day.</p>
<p>Also, if node has piqued your interest, I highly recommend checking out <a href="https://github.com/remy/nodemon">nodemon</a> , which basically looks at all your files under the directory and restarts the server on file changes automatically, so you don’t have to go back to the browser and press ALT+R like you are having a seizure.</p>
<p>Let me know if you have any questions and I will respond as quickly as I can because you are probably my only reader for this blog and I would hate to keep you waiting!</p>
<hr />
Equity Stock Predictor2014-04-02T00:00:00+00:00jasti.github.io/posts/stock<p>Do you want to know if there is a correlation between the news on a company and performance of the company’s stock in the market? Enter Stock Predictor - An open source project I authored that listens on Reuters via a twitter handle and screens the articles for positively or negatively correlated news for stocks you are interested in.</p>
<p>Based on the sentiment analysis of the news article, the program will suggest to take a direction on the stock. For example, if Reuters publishes a leak about Apple launching the iWatch and what a huge success it will become, the app will calculate a positive sentiment and mark it closer to 1. Depending on the exact sentiment score, the app will recommend buying a large number of stocks when the number gets closer to 1 and the app will recommend selling the stocks when the sentiment score is closer to 0. i.e shorting the stock.</p>
<p>You can find the project <a href="https://github.com/jasti/Stock-Predictor">here</a>.</p>
<p>Give it a spin, fork it and send me a pull request if you think something can be improved here. My vision for this a poor man’s high frequency trading app. All that this would be missing is a broker who is willing to give you electronic access actually trade electronically trade on a market. This type of access is called leased lines and firms spend quite a bit to have this kind of access. At the very least, you can simulate your paper profits of stocks you are interested in and who doesn’t like doing that!</p>
<hr />
Online Ads, The Elephant In The Room2014-03-21T00:00:00+00:00jasti.github.io/posts/block<p>Google made $50B in advertisement(Ad) revenue in 2013. Worth saying that again, FIFTY Billion. (Sure, when you compare that to the unbelievable doozy that FB made paying $19B for whatsapp, it doesn’t sounds like a lot.)
I have clicked (voluntarily) on one single add in all my waking years,( It was a really tempting locally made cookie Ad) but I don’t consider myself the average user either.</p>
<p>Having faith in the law of averages, I ran the numbers. With about 2.4B active internet users in the world and an average CPC( Cost Per Click) of 30 cents, each user must have clicked on roughly 62 adds in a year. Hard to believe that those clicks are contributing towards making a purchase decision.</p>
<p>As our devices’ screens get smaller, voice technology gets better, user awareness increases and the average users tolerance to ridiculous add filled pages decreases, there has to be a sharp decline in the number of clicks.I am not suggesting that Ad revenue reliant companies like Google and Facebook are in trouble, I’m sure they will figure other solutions to make money.</p>
<p>In the meantime, however, I don’t understand why most people don’t get Adblock software on their browsers? It is available for free as add ons to your browsers and it is completely legal to do so - at least right now. Who wouldn’t want a clean Ad free experience? Sure, some people might actually find those Ads useful, but maybe we could all use a smaller dose of consumerism.</p>
<p>I’m sure that you are super cool and know to go on the interwebs to look for the Adblock plugin using the Googles, but <a href="http://goo.gl/LhZNq">this</a> particular one deserves special mention if you are using Chrome, and if you are not using Chrome, START USING CHROME!</p>
<p>Gone will be the days you had to deal with ridiculous number of search engine ads or facebook ads. Twitter is sneaky with embedding ads within their tweets, but its just a matter of time that these Adblock software catch up.</p>
<p>Block on!</p>
<hr />
My New York Bohemian List2014-03-13T00:00:00+00:00jasti.github.io/posts/ny<p>There are many things I hate about New York but there are about twice the number of the things that I love about it. Here are a collections of things I tell people to do, eat and drink. Veers vegan/ vegetrian for the most part.</p>
<p>So here they are, my greatest hits:</p>
<h2>Do</h2>
<table>
<tbody>
<tr>
<td>Central Park</td>
<td>Run the bridal path or just anywhere<br /></td>
</tr>
<tr>
<td>Walk the Highline</td>
<td>Most beautiful in the spring or fall<br /></td>
</tr>
<tr>
<td>Museum of Modern Art</td>
<td>Indoor hanging helicopter is the highlight<br /></td>
</tr>
<tr>
<td>The MET</td>
<td>Remember, it’s donation based which they don’t make obvious<br /></td>
</tr>
<tr>
<td>Williamsburg, Brooklyn</td>
<td>Lot of cool little places in and around Bedford Ave<br /></td>
</tr>
<tr>
<td>Walk the Brooklyn Bridge</td>
<td>Ask for the ice-cream factory on the Brooklyn side<br /></td>
</tr>
<tr>
<td>Sleep No More</td>
<td>A moving play set in a warehouse that you walk around<br /></td>
</tr>
<tr>
<td>Gallow Green</td>
<td>A spooky rooftop bar with frequent visits by the casts of Sleep No More<br /></td>
</tr>
<tr>
<td>Google Office Rooftop</td>
<td>Find a Google buddy who works at Google<br /></td>
</tr>
</tbody>
</table>
<h2>Eat</h2>
<table>
<tbody>
<tr>
<td>Hangawi</td>
<td>Korean vegan food with unbelievable ambience <br /></td>
</tr>
<tr>
<td>Beyond Sushi</td>
<td>Best vegan sushi.<br /></td>
</tr>
<tr>
<td>Pio pio</td>
<td>Peruvian Chicken. Nandos bites this place’s dust<br /></td>
</tr>
<tr>
<td>Five Leaves</td>
<td>Australian cusines, done right (in Williamsburg, Brooklyn)<br /></td>
</tr>
<tr>
<td>Dirt Candy</td>
<td>Vegan prix fixe. Just brilliant<br /></td>
</tr>
<tr>
<td>Candle Cafe</td>
<td>Tasty vegetarian but nowhere close to healthy<br /></td>
</tr>
<tr>
<td>Bubby’s</td>
<td>American Southern food and they have the best pies in town<br /></td>
</tr>
<tr>
<td>La Esquina</td>
<td>Mouth watering tacos. Enough said <br /></td>
</tr>
<tr>
<td>Sigiri</td>
<td>If you’ve never had Srilankan, just go. Order the Lamprais<br /></td>
</tr>
<tr>
<td>Kottu House</td>
<td>If you liked Sigiri, go here for the Kottu. Say hi to Chaz<br /></td>
</tr>
</tbody>
</table>
<h2>Drink</h2>
<table>
<tbody>
<tr>
<td>Amor y Amargo</td>
<td>A Bitters Tasting Room<br /></td>
</tr>
<tr>
<td>Greenpoint Brewery</td>
<td>They experiment about 300 beers a year<br /></td>
</tr>
<tr>
<td>230 5th Ave</td>
<td>Expensive, touristy but also probably the best rooftop bar in the city<br /></td>
</tr>
<tr>
<td>The Raines Law Room</td>
<td>Probably the best prohibition setting bar in NYC with solid cocktails<br /></td>
</tr>
<tr>
<td>Chaan tea house</td>
<td>A place to wind down with a book or meet friends for tea<br /></td>
</tr>
<tr>
<td>Blue Bottle Coffee</td>
<td>Rockefeller building<br /></td>
</tr>
<tr>
<td>Toby’s Estate Coffee</td>
<td>Williamsburg, Brooklyn<br /></td>
</tr>
<tr>
<td>Kaffe 1668</td>
<td>They take their coffee very seriously here<br /></td>
</tr>
<tr>
<td>Le Colombe</td>
<td>Mochas are really good<br /></td>
</tr>
<tr>
<td>Mud Truck</td>
<td>Try the mocha from the truck or the store<br /></td>
</tr>
<tr>
<td>Round K</td>
<td>Murakami would go here for coffee <br /></td>
</tr>
</tbody>
</table>
<h2>Sweets</h2>
<table>
<tbody>
<tr>
<td>Lady M</td>
<td>Haven’t had anything like it before<br /></td>
</tr>
<tr>
<td>Morgenstern’s Icecream</td>
<td>The icecream temperature they serve at is perfect<br /></td>
</tr>
<tr>
<td>Levain</td>
<td>The best cookie in the world (Dark Chocolate peanut butter)<br /></td>
</tr>
<tr>
<td>Magnolia Cupcakes</td>
<td>Quite touristy now, but that shoudn’t stop you<br /></td>
</tr>
<tr>
<td>Momofuku milk bar</td>
<td>Get the compost cookie<br /></td>
</tr>
<tr>
<td>Van Leeuwen</td>
<td>Natural ice-creams. Try the Ginger or Gianduja<br /></td>
</tr>
</tbody>
</table>
<p><br /></p>
<p>I’ve done my fare bit of travelling and I hate to admit it, but New York is the world’s most amazing city and these places are just a few reasons for it.</p>
<hr />
Running Off A Cliff2014-02-01T00:00:00+00:00jasti.github.io/posts/marathon<p>I was on vacation visiting my brother’s in Texas. After a heavy meal and
slowly drifting into an incredible food coma, I picked up a book lying next to bed on a shelf called <a href="http://www.amazon.com/Four-Months-Four-hour-Marathon-Updated/dp/0399532595">4 hour marathon</a>.
I was curious because I felt that the entire idea of running a marathon seemed mental, let alone a 4 hour one.
It was a tiny book and I finished it in one sitting. I put the book back on the shelf and forgot all about it, but little did I know that the <a href="http://www.youtube.com/watch?v=imamcajBEJs">seed</a> was planted.</p>
<p>I hated running. I didn’t think of running a marathon as humanly possible, unless you had a certain type of body and especially not, if you weighed above 200 pounds.
Before then, the farthest i had ever run in my life was 3 miles and I could count the number of times
I did that on my fingers. I’d rather watch <a href="http://www.youtube.com/watch?v=kfVsfOSbJY0&feature=kp">this</a>, 30 times in a row than run 3 miles.</p>
<p>A couple of months after reading the book, my girlfriend at the time (now fiancee), brought up the topic of running a marathon. She said that it was, physically, one of the hardest things she had ever done.
When a girl you like and have been dating for 3 months says such a thing, of course you act irrationally and tell her that it’s a great idea!
We signed up for the marathon with exactly 4 months to go. I can’t tell you how many times I
regretted that decision I made that day during our training sessions, but it’s all worth it when you go across that finish line and have just done something you thought was completely
impossible just 4 months ago.</p>
<p>It’s great to know your limits, but I think most often you just need to jump off the cliff and build your parachute on your way down.
Yes, it’s not going to be pretty all the time, but the marginal value of every success is so high to the marginal loss of failure for outlandish goals.
Now go sign up for that marathon and when you finish - scream, curse and cry.</p>
<hr />