Wednesday, September 21, 2016

Generator functions. Why and when should we use them.

You probably have heard of them. But as a beginner python programmer, we tend to ignore it's values because we fail to find uses of it. When we have loops and itertools and other fancy toys offered, why should we bother to understand them. If you want to take your programming  or understanding of Python to the next level, you must understand and use generators more and more.

Before we start, I want to make a simple program that prints square of (hence generates and delivers) 5 numbers from 1 to 5. So what is the first idea that pops up ? Let's use a for loop.


def square(N):
    for i in range(1, N+1):
        print i**2

>>>square(5)
1
4
9
16
25

Let's say, you need all the values generated for some other calculation. I know, above example is too lame . But imagine after some complicated calculations, you have generated some result.
So you probably will return all values together in an array. And to that, the first thought in our minds is, using a LIST. Let's use it. 

Now our function will look like this.

>>> def my_function(N):

    new_list = []

    for i in range(1, N+1):

        new_list.append(i**2)

    return new_list




>>> my_function(5)
[1, 4, 9, 16, 25]


All this looks easy. You probably are thinking...If generators are going to ruin this idea, I don't need it. Think again.
  1.  This LIST business will soon start becoming a nuisance as the  number of such calculations and variables increase (believe  me...thinking of new names for variables... SUCK).
  2.  What if you don't know your requirement. How many results  do you want.?
  3.  You had to create an array and it will take memory space  which becomes bigger with length of the list and size of each  element.
  4. Once you are dealing with very large list (imagine a million), generating a list of 1 million (let's assume integers) will take around 300 mb in memory.
A generator will make it all...Simpler. Here's the same example with generator function.


def square(N):
    for i in range(1, N+1):
        yield i**2

gen = square(5)

print gen.next()
print gen.next()
print gen.next()
print gen.next()
print gen.next()
print gen.next()

Result:

1
4
9
16
25

Traceback (most recent call last):

  File "D:\mydrive\Desktop\PyScripts\generators.py", line 14, in <module>
    print gen.next()
StopIteration
>>>

You can guess the reasons for the last traceback. We got out of fuel. In order to avoid this, you can always use generator object to iterate.


def square(N):
    for i in range(1, N+1):

        yield i**2

gen_object = square(5)

for result in gen_object:

    print result

Result:

1
4
9
16
25


More fun with generators:

You can also make generator functions from list comprehensions. Use parenthesis instead of square bracket to make a gen function.


l = (x**2 for x in range(1,6))  

print l
for value in l:
    print value


Result:

<generator object <genexpr> at 0x0000000002D50F78>
1
4
9
16
25

Thursday, September 15, 2016

Python: Running executables using subprocess module

There are many ways to run a bat/exe file but the most widely used is the subprocess module.
And I would like to clearly show, how exactly to do that. When should you use what arguments.
I am going to specifically talk about the check_output module .

Note : This function is used when we want the output of the bat file collected inside a variable.

Condition 1: The exe file is in the same location as of the python script that wants to run it.

Explanation: 
You are in a Folder > MyPrograms  . And the bat/exe file lies inside this folder. (Refer picture below)

Single folder with bat and python files

Content of the hello.bat file


@echo off
echo "This is a test bat"


My script contents:


import subprocess
result = subprocess.check_output('hello.bat')
print result


Now if you run it , the result will be a:

"This is a test bat"

This is easy. Lets look at another condition.



Condition 2: The exe file is in a different location from the python script that wants to run it.

Two diff folders with python script and bat file

No change in contents of hello.bat file. Some change in myscript.py file.


import subprocess
result = subprocess.check_output('hello.bat', shell=True, cwd=r'D:\inchowar\Desktop\temp')
print result


The trick here is, cwd (current working directly) argument helps you automatically switch switch to that location specified before it tries to find the bat file . 

Otherwise you would have to do it yourself . Change directory before you run the bat file using os.chdir(r'D:\inchowar\Desktop\temp'), and then run the bat file, Obviously, this becomes tedious.

Note: When you are using the cwd argument, the bat file should not have a path before it. Only the name of bat file i.e hello.bat. NOT c:\Temp\hello.bat

As to when to use shell=True, I think this article has much better explanation than I can provide here.

Thanks for reading.

Monday, September 12, 2016

Regular Expressions with Python: Look ahead and Look behind

These two important tools have been bugging me every time I come up with a situation. Because I never remember this one. Blame my memory.
So it's time I cement this idea somewhere . And no better place than a blog page.

LOOK AHEAD

LOOK AHEAD means , I want to see what lies ahead first, and then decide, to do something now.


Syntax of regex pattern:  pattern1(?=pattern2)


Where Pattern1 is the pattern for the part that we ACTUALLY want to capture. Pattern2 is the pattern which needs to be found as MANDATORY. Logically speaking, 

IF PATTERN2 is FOUND, then print/get/capture/show PATTERN1


Example: Let there be a string "Hello World"

Aim: I want to find 'Hello' ...ONLY IF its followed by 'World'



re.search(r'\w+(?= World)', 'Hello World').group()

Result : >> 'Hello'

Explanation:

  •  The \w+ is Pattern1
  •  The 'World' is Pattern2
  •  r'\w+(?= World)' means, Find anything which fits \w+ IF it is followed by 'World'

Rules:
  • The Pattern2 needs to be in parenthesis.
  • If pattern2 itself has parenthesis, then they need to be bracketed. i.e [ ( ] or [ ) ]

LOOK BEHIND


Its just the opposite of the above. It means , I want to see what lies behind me , and then decide to do something.



Syntax of regex pattern:  (?<=pattern1)pattern2


Where pattern2 is the pattern for the part that we ACTUALLY want to capture. Pattern1 is the pattern which needs to found as MANDATORY.


IF PATTERN1 is FOUND, then print/get/capture/show PATTERN2


Example : We will take the same example. 'Hello World'

Aim : I want to find 'World' ONLY if it is preceded by 'Hello ' (The space here also counts)

>>> re.search(r'(?<=Hello )\w+', 'Hello World').group()

Result : 'World'

Rules: 
  • Pattern1 needs to be in parenthesis
  • If pattern2 itself has parenthesis, then they need to be bracketed. i.e [ ( ] or [ ) ]


LOOK AHEAD & LOOK BEHIND COMBINED

Consume only if it is surrounded by the things we want.



Syntax of regex pattern: (?<=pattern_behind)pattern_middle(?=pattern_ahead)


Where pattern_middle is the pattern for the part that we ACTUALLY want to capture. Pattern_behind and Pattern_ahead are patterns which need to be found as MANDATORY.


IF PATTERN_AHEAD AND IF PATTERN_BEHIND are BOTH found, consume PATTERN_MIDDLE


Example: We will take a new example . 'Hello My World'

Aim: I want to find any word that occurs in between Hello & World


>>> re.search('(?<=Hello )\w+(?= World)', 'Hello My World').group()

Result : 'My'


Wednesday, September 7, 2016

Kerala : How to plan for a lifetime's experience

I have been planning this for months and I finally did it. A full stretch Kerala trip with Kanyakumari as an added bonus. I will not tell you what to do, but how to do it. How to plan your dream Kerala Trip. Caution: Mine was too comprehensive. Needs high energy and will be quite tiring. 

I chose the monsoon theme..I wanted to be at the middle of the action. This is the only place in India which offers monsoon as a tourism opportunity. I was lucky. I expected super heavy rains. Instead. I ended up right in between the rainy season and summer which was Perfect.

Advantages in travelling in mid August

  • Rainfall almost over.
  • Waterfalls at their fullest.
  • Greenery all over the state.
  • Backwaters, beach sides and rivers all very clean.
  • Abundant coconuts to enjoy.
  • Hotel rates brought down.
  • Less crowd everywhere.
  • Festive Preparation going on.
  • Shopping becomes pocket friendly.
Our route map:







                                                   

Day 1
Day 2
Day 3
Day 4
Day 5
Day 6
Day 7
Day 8
Day 9
Panvel
Trivandrum
Kovalam
Kovalam
Thekkady
Munnar
Munnar
Kochi
Pune

Why should you go by train ?

Konkan railways is different from all others. Amazing sights . Beautiful waterfalls (some of them were huge). Lots of tunnels. Breathtaking rivers and bridges. Glimpses of the Arabian sea every now and then. It's a journey everyone should make at least once in his lifetime. (We took the Sampark Kranti from Panvel)

Our Journey through photographs

Kovalam Beach Side

Lighthouse standing guard at Kovalam Beach
Walking along the beach is such a joy

Sunset at Kovalam Beach



Alleppey Beach beside our resort
Sunset at Alleppy beach

A tiny boat with the sunset in background..Just amazing view
What's Kerala without Coconuts

Kerala offers various kinds of Coconuts. The water and the pulp inside are both consumed
Different kinds of coconuts can be found everywhere.

Kanyakumari

Waiting for sunrise at Kanyakumari was such a unique experience..One side Indian Ocean, another side..The Arabian Sea
Poovar Backwaters

Poovar Backwater was a good experience.

Thekkady/Periyar

An hour long boat ride in the forest. We saw an elephant herd.
Periyar Wildlife Sanctuary
Munnar 

Munnar Tea Gardens

Kochi beach with Chinese fishing nets



The harbor at Kochi is a busy one. You can see huge ships crossing it, all day and night.

The oil industry can be seen at the right corner. Ships keep crossing every now and then at Kochi Harbor

We finally got caught in rain at Kochi . It nevertheless made the scene more captivating.

Huge ship entering , while a couple leisurely walks by.

Very Important Tips



Before you journey starts
  • Book for trains from the list of Konkan Railways that suits best for you. (You can refer this link)
  • Book your train at least 2 months before. There's plenty of rush in these trains.
  • Plan your trip from the bottom of Kerala to the top (Hence,  book train till Trivandrum i.e Ernakulam Railway Station) . Catch return flight from Kochi. 
  • Book the train and flight YOURSELF. DO NOT let a travel company book them for you.
  • For each city you are going to visit, note down good hotels fitting your budget from good websites like TripAdvisor or MakeMyTrip. Then book your vacation from makemytrip or similar travel organizer. Tell them that you have already bought tickets. They need to plan according to you. Also handover the list of hotels , so they book only from your chosen list.


What to Pack (The most important things)

  • Rain gear (Umbrella, raincoat, lots of plastic bags)
  • Rain proof Foot gear (My Quechua all weather sandals were enough for the whole tour.), Extra bathroom flip flops for beaches.
  • Extra foldable bag to bring back all shopped items.
  • Laptop (A DSLR is going to need constant flushing of photos)
  • One light jacket (Munnar might be chilly)
  • Towel, Swimming gear, Cap.
  • Binocular.

Locations


Trivandrum:

Visit the Shangumugham Beach same day as you arrive, at evening. Preferably, book the "Uday Suites" hotel (Book online. Use as much vouchers as possible) . From there, it's walking distance to the beach.

Kanyakumari: 

Plan your Kanyakumari trip from Kovalam. Start at night 2 AM , so you can reach before Sun Rise.

Backwater experience:
We suggest, you visit the Poovar Backwater cruise. It's worthy, very private and beautiful. Visit this link for an idea.
Beware: Bargain very well. Take the 1 hr trip ONLY. Any longer is not worth. Pay only max 1000rs per head. No more.

Alleppey:
Choose for a resort along the beach side. Let it be far from the city. You will love the stay. Nothing but clean sea shores, beautiful sun sets. And the sound of the roaring sea. (Please carry mosquito repellents too).

Suggestion: If possible, do not choose to live on a houseboat. I heard it is highly boring and costly. Not worth it. It's your choice. 
Note : You can still visit them for some time, if not stay.

Munnar:
Choose a hotel with a great view. Read review comments before you book. 
Visit the Tata Tea factory. (There's nothing to eat there, so have your breakfast before you start).

Thekkady:
While in the city, choose a hotel in the midst of the market. Stroll around the market and you will get an idea of the richness of the local market in spices, chocolates, and art. The Martial Arts and Dance show is also nearby centrally placed. Shows start at evening . Around 200 rupees per head per show. Don't miss that.

Kochi:
Stay at a hotel near the sea beach. It feels european. It's fun to walk around the coast. Lots of classy hotels to dine. Also check timings for all visiting spots (churches, museum or zoo are closed on some odd days). While going to airport, ask the driver to cross the sea on steamer (the car itself rides on it).

Shopping

This might be the favourite part of many people. Kerala has lots to look out for. 

Saaris and other clothings: From Kochi, or Trivandrum
Chocolates: From Thekkady and Munnar.
Banana chips: Anywhere.
Spices: Thekkady and Munnar
Tea: From munnar local market.
Artifacts: They are very costly. So bargain well.

In all, enjoy the most. Eat, explore, sit beside the shore, and gaze. Bon Voyage.

Kovalam Beach road. Watch the amazing sunset.


Thursday, July 14, 2016

Regular Expressions with Python: Examples of common functions

Regular Expressions are tools to filter and grab information from a heap of strings. Python has got a lot of toys to deal with it. The module "re". I am going to give you a quick ride through it. The ones that we use/gonna use the most.

So just immediately launch your favourite IDE and type "import re". Let's get started.

re.search:


for the object r where r = re.search(pattern, subject),

r.start() - Gives the start point where the match was found.
r.end() - Gives the end point where the match was found.
r.group(0) - Returns the full matched string
r.group(1) - Returns the matched string for the 1st group
r.group(2) - Returns the matched string for the 2nd group

And so on....

Note : r.group() is same as r.group(0)

The number of groups available depends on the number of groups used in your pattern.

Example:


>>> regex = r"([a-zA-Z]+) (\d+)"
>>> re.search(regex, 'July 4')
<_sre.SRE_Match object at 0x0000000002E4D140>
>>> r = re.search(regex, 'July 4')
>>> r.end()
6
>>> r.start()
0
>>> 'July 4'[0:6]
'July 4'
>>> 'July 4'[0:5]
'July '
>>> r.group(0)
'July 4'
>>> r.group(1)
'July'
>>> r.group(2)
'4'
>>> r.group(3)

Traceback (most recent call last):
  File "<pyshell#12>", line 1, in <module>
    r.group(3)
IndexError: no such group

re.findall


>>> r = re.findall(regex, 'July 4 August 10 Sept')
>>> r
[('July', '4'), ('August', '10')]

Same example with findall...observe how we got two results since there are two patterns in the string .

Noteworthy : If you are using groups with findall, then the result will be a list containing tuples as above.
But if you are just using single regex (no grouping), the result will be a list with strings .

Example:


>>> re.findall('\w+', 'July 4 August 10')
['July', '4', 'August', '10']

re.finditer

The finditer returns an iterator of re.MatchObjects.


>>> it = re.finditer('\w+', 'July 4 August 10')
>>> for i in it:
        print 'July 4 August 10'[i.start():i.end()]

>>> for i in it:
       print i.group()

 
July
4
August
10

Tuesday, February 2, 2016

Rajasthan: The Best backpacking destination in India

Why Rajasthan?

  • Suitable for all size of plans . Week long , month long , or a day long.
  • Numerous popular cities to visit.
  • Shopper's paradise.
  • Best forts in India in one State.
  • Diversity - Greenery, Desert, Lakes, Urbanization or Tranquility. You will get it all.
  • Wildlife Sanctuaries
  • Heritage cities - Best for people who love cities with a history.
  • Food - A host of popular dishes to cherish. Vegetarians rejoice.
  • Transport - All options are available, abundant and safe. Trains. buses or airways.
  • Budget friendly - One can choose a budget fitting as he likes. Everyone can enjoy irrespective of his pocket strength.

The Plan


                              


  Day 1
    Day 1
      Day 2 
   Day 3
   Day 4
   Day 5
   Day 6
Day 7
  Pune
 Mumbai
  Chittorgarh
 Udaipur
  Jodhpur
  Jodhpur
 Jaisalmer
Jaisalmer



































1.  Pune to Mumbai (Bandra Terminus) (via Bus) , Distance: 159 km

2.  Mumbai to Chittorgarh (via Train 22901 / UDZ SUPER FAST) Distance: 831 km

   Duration: One night stay in Chittorgarh

3.  Chittorgarh to Udaipur (via Passenger Train) , Distance: 117 km

   Duration:  Same day departure for Jodhpur

4.  Udaipur to Jodhpur (via State transport Bus) , Distance : 261 km

  Duration: Two night stay at Jodhpur

5.  Jodhpur to Jaisalmer (via Bus) , Distance : 285 km

 Duration: Two night stay at Jaisalmer (First night at Sam Sand Dunes)

 

PLAN DETAILS

Before you get excited about reaching Rajasthan , you must be aware that your Trip began as soon as you left your room. Ours started as soon as we left Pune and caught our bus to Mumbai. This was a good opportunity for me to spend some time in the City Of Dreams.Mumbai.  Our train was to leave at night from Bandra Terminus. So we ventured wherever our hearts took us.

Mumbai Special Thali























Chittorgarh:
The largest fort in India - Chittorgarh Fort
Prakash Palace
Prakash Palace
Prakash Palace Ruins
Padmini Temple



Inside the fort:
  • 7 Gates
  • 2 Stambhas
  • 3 Palaces
  • 1 temple

Note>  Time needed to cover all points: Minimum 4 hours


What to do :
  • Arrive early in the morning and check-in to any hotel (should be closer to the fort) 
  • Hire an auto and visit the Fort. Take plenty of time to explore. It is huge enough for a full day.


Udaipur : The city of lakes

Although we spent just one day here, I would suggest, you should plan for at least 2 days (1 night halt), as this city has a lot to offer. I won't list them all. But for back packers with a short time, these location are the ones I would recommend as a "must visit".
  • City Palace
  • Lake Palace (Unfortunately, you can just enjoy view from a distance as tourists are no more allowed to visit it)
  • Lake Pichola

Note : 1) For all the above spots , the good thing is, you don't need to travel at different parts of the city. All are located around each other at walking distances.


2) Do not miss the evening light & sound show in City Palace (It's ticket are to be bought separately).

Lake Palace
City Palace

City Palace Light and Sound show
City Palace Evening Light Show

What to do:
    • Take a passenger train in the morning from Chittorgarh Station. It's about 3 hrs journey.
    • Visit your selected spots in Udaipur (Depends on how many days you have. There is a lot to see here). 
    • Watch the evening light and sound show in City Palace.
    • Shop on City palace road.
      Shops in Udaipur around City Palace
      Shops around City Palace Gate
    • Catch a night bus from City Bus Station for Jodhpur.


      Jodhpur : The Sun city


      Jodhpur is the second largest city in Rajasthan. So it is abound with markets, food and what not. This is where you stop to shop. We spent 2 days here. Most amazing looking fort I have ever seen. The Mehrangarh fort. It looks like a giant sitting on a mountain. 
      For shoppers, busiest market near Clock Tower. It's a must visit (women are going to love this).

      What to do :
      • Get a hotel near the Mehrangarh Fort (We stayed at Hotel Haveli ) and the room with a view of the fort. 
        Room in Hotel Haveli in Jodhpur
      • Go on a hike from your hotel to the fort. Explore all day.
      • At evening , visit the city market near the clock tower.
      • Try a variety of cuisines offered in road side hotels. The tea tastes awesome. I discovered a host of sweet dishes.
      • Have dinner at roof top with a view of the Mehrangarh Fort alive with lights. Most amazing view of any fort you have ever seen.


        Checkout at morning and leave for Jaisalmer via Bus. The buses are another thing to notice. The interiors are decorated to remind you of the culture and art of Rajasthan .




        Jaisalmer: 


        Our final destination. The golden city of India. Jaisalmer . And it's crown jewel,
        Jaisalmer fort also known in other languages as Sonar Kella, Sone Ka Kila. Jaisalmer fort is the only fort in India where people still reside. It has around 5000 population. The fort is self sustaining . It has got everything you can imagine. Temples, Market, Food, post office. There is nothing like it in the whole world.
                                    I bet you must be thinking..When do we get to see some sand . Well....Its all sand here. The fort itself is build with Sand Stones. Even cooking utensils are made out of them . Also fossil stones are abundantly found which are used to make a variety of things. I leave you to discover all that by yourself.


        What to do :




          • You will reach Jaisalmer by afternoon. Have lunch.
          • Then book a trip and night stay in Sam Sand Dunes. Don't pay more than 2000Rs for one person.
          • Enjoy ride on Jeep over Sand Dunes. Enjoy the Thar desert. Explore yourself around.
          • At night, the camp comes alive. There is Rajasthani dance and music.

            Rajasthani Folk Song

              • Next day,come back to Jaisalmer city in the morning and check in to a hotel outside the fort, conveniently located at walking distance. (We stayed at 1st Gate hotel. The best hotel experiences of my life. Little overpriced but worth every penny.)
              • Go to explore the fort. Visit the market around. (I explored the fort at least 3 times...The fort is too enticing to visit. Loved the slim lanes, homes, shops, Puppets hanging from the walls...I just can't get enough)
              • Next day, checkout to catch your train :( .

                Say your good byes to the most amazing back packing trip you ever had.

                Total Pageviews