Pandas Introduction

by Niranjan B Subramanian

In this article, we’ll discuss Pandas, which is the most popular python data analysis library.

Data analysis is a process of cleaning, exploring, organizing, describing, and visualizing data.

Pandas is mainly used for cleaning and exploring the data.

Two primary data structures in pandas are: Series and DataFrame both built on top of numpy.

PANDAS SERIES:

A pandas series is a one-dimensional array that contains a sequence of values. It’s like a Numpy array, but it differs by having an index. Unlike Numpy array, which is homogeneous, pandas series can hold values of different data types.

Initializing Pandas Series:

We can initialize the pandas series in multiple ways. The most common way is to use python containers such as lists or dictionaries.

import pandas as pd
import numpy as np

s = pd.Series([4,7,4,2,6])
print(s)

import pandas as pd

import numpy as np

s = pd.Series([4,7,4,2,6])

print(s)

OUTPUT:
0    4
1    7
2    4    
3    2
4    6
dtype: int64

OUTPUT:

0 4

1 7

2 4

3 2

4 6

dtype: int64

The output consists of two columns. The first is the index, while the second is the values.

dtype: int64 denotes that the data type of the values in the Series is int64.

Since we did not explicitly specify an index, a default index [0 …… N-1] will be created where N is the length of the data.

The following code will create a series with a customized label.

s = pd.Series([4,7,4,2,6], index=['a','b','c','d','e'])
print(s)

1 2	s = pd.Series([4,7,4,2,6], index=['a','b','c','d','e']) print(s)

OUTPUT:
a    4
b    7
c    4
d    2
e    6
dtype: int64

OUTPUT:

a 4

b 7

c 4

d 2

e 6

dtype: int64

Let’s see how to create a series using the python dictionary.

d = {'a':4, 'b':7, 'c':4, 'd':2, 'e':6}
s = pd.Series(d)
print(s)

d = {'a':4, 'b':7, 'c':4, 'd':2, 'e':6}

s = pd.Series(d)

print(s)

OUTPUT:
a    4
b    7
c    4
d    2
e    6
dtype: int64

OUTPUT:

a 4

b 7

c 4

d 2

e 6

dtype: int64

When using a dictionary, the keys of the dictionary are used as the index labels

You can also create a series using numpy functions

arr = np.array(['a','b','c','d','e'])
s = pd.Series(arr)
print(s)

arr = np.array(['a','b','c','d','e'])

s = pd.Series(arr)

print(s)

OUTPUT:
0    a
1    b
2    c
3    d
4    e
dtype: object

OUTPUT:

0 a

1 b

2 c

3 d

4 e

dtype: object

s = pd.Series(np.arange(2,6))
print(s)

1 2	s = pd.Series(np.arange(2,6)) print(s)

OUTPUT:
0    2
1    3
2    4
3    5
dtype: int32

OUTPUT:

0 2

1 3

2 4

3 5

dtype: int32

s = pd.Series(np.linspace(0,10,5), index=list('abcde'))
print(s)

1 2	s = pd.Series(np.linspace(0,10,5), index=list('abcde')) print(s)

OUTPUT:
a     0.0
b     2.5
c     5.0
d     7.5
e    10.0
dtype: float64

OUTPUT:

a 0.0

b 2.5

c 5.0

d 7.5

e 10.0

dtype: float64

We can examine the index of the series using the property index, and the values can be retrieved using the property values.

print(s.index)
print(s.values)

1 2	print(s.index) print(s.values)

OUTPUT:
Index(['a', 'b', 'c', 'd', 'e'], dtype='object')

[ 0. 2.5 5. 7.5 10. ]

OUTPUT:

Index(['a', 'b', 'c', 'd', 'e'], dtype='object')

[ 0. 2.5 5. 7.5 10. ]

Peek into Data:

By using the head(), we can peek the data at the beginning. By default, it returns the first five rows of the data.

s = pd.Series(np.linspace(0,16,9), index=list('abcdefghi'))
print(s.head())

1 2	s = pd.Series(np.linspace(0,16,9), index=list('abcdefghi')) print(s.head())

OUTPUT:
a    0.0
b    2.0
c    4.0
d    6.0
e    8.0
dtype: float64

OUTPUT:

a 0.0

b 2.0

c 4.0

d 6.0

e 8.0

dtype: float64

We can also specify the number of rows. For example, the following code will print the first three rows of the data.

print(s.head(3))

1	print(s.head(3))

OUTPUT:
a    0.0
b    2.5
c    5.0
dtype: float64

OUTPUT:

a 0.0

b 2.5

c 5.0

dtype: float64

Similarly, we can peek the last five rows of our data using the tail() method.

print(s.tail())

1	print(s.tail())

OUTPUT:
e     8.0
f    10.0
g    12.0
h    14.0
i    16.0
dtype: float64

OUTPUT:

e 8.0

f 10.0

g 12.0

h 14.0

i 16.0

dtype: float64

To count the number of elements in the series, we can use the size or shape attribute.

print(s.size)
print(s.shape)

1 2	print(s.size) print(s.shape)

OUTPUT:
9
(9,)

OUTPUT:

(9,)

Alternatively, we can also use the len function.

print(len(s))

1	print(len(s))

OUTPUT:
9

OUTPUT:

The data in the pandas series can be accessed in two ways: either through their index labels or through their numerical position.

The following code demonstrates how to access the value using index labels.

print(s['a'])

1	print(s['a'])

OUTPUT:
0.0

1 2	OUTPUT: 0.0

We can also access multiple row values by passing a list of index labels.

print(s[['a','c','e']])

1	print(s[['a','c','e']])

OUTPUT:
a    0.0
c    4.0
e    8.0
dtype: float64

OUTPUT:

a 0.0

c 4.0

e 8.0

dtype: float64

PANDAS DATAFRAME:

A DataFrame is a two-dimensional data structure made up of rows and columns.

Unlike Series, which had an Index array with associated labels for each element, a DataFrame has two indexes: a column index and a row index.

We can create a DataFrame in multiple ways. The following code creates a DataFrame using a dictionary of lists.

index = ['Uttar Pradesh','Tamil Nadu', 'Karnataka', 'Rajasthan', 'Gujarat', 'Maharashtra', 'Punjab', 'Assam', 'Kerala']
df = pd.DataFrame( { \
    'Population(Millions)':[223.9,79.10,61.13,75.98,67.60,123.17,30.05,31.17,34.52], \
    'GDP(lakh crore)':[15.42,17.25,15.10,9.24,11.62,24.96,5.18,2.54,6.87]}, index=index)

print(df)

index = ['Uttar Pradesh','Tamil Nadu', 'Karnataka', 'Rajasthan', 'Gujarat', 'Maharashtra', 'Punjab', 'Assam', 'Kerala']

df = pd.DataFrame( { \

'Population(Millions)':[223.9,79.10,61.13,75.98,67.60,123.17,30.05,31.17,34.52], \

'GDP(lakh crore)':[15.42,17.25,15.10,9.24,11.62,24.96,5.18,2.54,6.87]}, index=index)

print(df)

OUTPUT:
                      Population(Millions)    GDP(lakh crore)
Uttar Pradesh                223.90            15.42
Tamil Nadu                    79.10            17.25
Karnataka                     61.13            15.10
Rajasthan                     75.98             9.24
Gujarat                       67.60            11.62
Maharashtra                  123.17            24.96
Punjab                        30.05             5.18
Assam                         31.17             2.54
Kerala                        34.52             6.87

OUTPUT:

Population(Millions) GDP(lakh crore)

Uttar Pradesh 223.90 15.42

Tamil Nadu 79.10 17.25

Karnataka 61.13 15.10

Rajasthan 75.98 9.24

Gujarat 67.60 11.62

Maharashtra 123.17 24.96

Punjab 30.05 5.18

Assam 31.17 2.54

Kerala 34.52 6.87

Now let’s see how we can create a DataFrame using a numpy array.

df = pd.DataFrame(np.arange(12).reshape(4,3))
print(df)

1 2	df = pd.DataFrame(np.arange(12).reshape(4,3)) print(df)

OUTPUT:
    0   1   2
0   0   1   2
1   3   4   5
2   6   7   8
3   9  10  11

OUTPUT:

0 1 2

0 0 1 2

1 3 4 5

2 6 7 8

3 9 10 11

Practically we are less likely to use python containers to create a DataFrame. We often use pandas data reading methods to create a DataFrame.

Pandas have an awesome array of data readers for various types of formats, from CSV through SQL databases to pickle files.

The following code will read a CSV file and save it as a pandas DataFrame object.

You can download the dataset here.

df = pd.read_csv('data.csv')

1	df = pd.read_csv('data.csv')

We can quickly peek at the beginning of the data by using head()

df.head()

df.head()

OUTPUT:
     State	                        Population(Millions)    Literacy Rate(%)    No of Schools  
0    Andaman and Nicobar Islands	0.38                    86.27               414
1    Andhra Pradesh	                84.67                   67.40               62702
2    Arunachal Pradesh	            1.38                    66.95               4047
3    Assam	                        31.17                   73.18               71042
4    Bihar	                        119.24                  63.82               84236

OUTPUT:

State Population(Millions) Literacy Rate(%) No of Schools

0 Andaman and Nicobar Islands 0.38 86.27 414

1 Andhra Pradesh 84.67 67.40 62702

2 Arunachal Pradesh 1.38 66.95 4047

3 Assam 31.17 73.18 71042

4 Bihar 119.24 63.82 84236

To get the number of rows and columns, we can use the shape attribute.

The shape attribute returns a tuple in which the first value is the number of rows, and the second number is the number of columns.

print(df.shape)

1	print(df.shape)

OUTPUT:
(36, 4)

1 2	OUTPUT: (36, 4)

The names of the columns and its type in a DataFrame can be accessed through the columns and dtypes property.

print(df.columns)
print(df.dtypes)

1 2	print(df.columns) print(df.dtypes)

OUTPUT:
Index(['State', 'Population(Millions)', 'Literacy Rate(%)', 'No of Schools'], dtype='object')

State                    object
Population(Millions)    float64
Literacy Rate(%)        float64
No of Schools             int64
dtype: object

OUTPUT:

Index(['State', 'Population(Millions)', 'Literacy Rate(%)', 'No of Schools'], dtype='object')

State object

Population(Millions) float64

Literacy Rate(%) float64

No of Schools int64

dtype: object

Similarly, the indexes are accessible via the index property

print(df.index)

1	print(df.index)

OUTPUT:
RangeIndex(start=0, stop=36, step=1)

1 2	OUTPUT: RangeIndex(start=0, stop=36, step=1)

Getting info about your data:

We can get more information about our data by using the info method.

It will provide the number of rows, names, and types of your columns and memory usage of our data frame.

print(df.info())

1	print(df.info())

OUTPUT:
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 36 entries, 0 to 35
Data columns (total 4 columns):
State                   36 non-null object
Population(Millions)    36 non-null float64
Literacy Rate(%)        36 non-null float64
No of Schools           36 non-null int64
dtypes: float64(2), int64(1), object(1)
memory usage: 1.2+ KB
None

OUTPUT:

RangeIndex: 36 entries, 0 to 35

Data columns (total 4 columns):

State 36 non-null object

Population(Millions) 36 non-null float64

Literacy Rate(%) 36 non-null float64

No of Schools 36 non-null int64

dtypes: float64(2), int64(1), object(1)

memory usage: 1.2+ KB

None

To get a summary of some basic statistics like percentile, mean, standard deviation, etc. we can use the describe method.

print(df.describe())

1	print(df.describe())

OUTPUT:
            Population(Millions)  Literacy Rate(%)  No of Schools
count             36.000000         36.000000          36.000000
mean              37.213889         78.302778       42287.388889
std               48.436748          8.267518       53435.898253
min                0.060000         63.820000          45.000000
25%                1.440000         70.937500        3575.250000
50%               22.115000         78.480000       23170.500000
75%               62.747500         86.287500       64601.500000
max              223.900000         93.910000      255969.000000

OUTPUT:

Population(Millions) Literacy Rate(%) No of Schools

count 36.000000 36.000000 36.000000

mean 37.213889 78.302778 42287.388889

std 48.436748 8.267518 53435.898253

min 0.060000 63.820000 45.000000

25% 1.440000 70.937500 3575.250000

50% 22.115000 78.480000 23170.500000

75% 62.747500 86.287500 64601.500000

max 223.900000 93.910000 255969.000000

Change Index:

By default the index is incremental integers, starts with 0 and continues until n-1.

However, you can change the index to be one of your columns by using set_index.

In the following code, we use the set_index method to set our DataFrame’s index to the state column.

df.set_index('State', inplace=True)

1	df.set_index('State', inplace=True)

OUTPUT:
                              Population(Millions)	Literacy Rate(%)	No of Schools
                      State			
Andaman and Nicobar Islands	    0.38	                 86.27	                414
Andhra Pradesh	                84.67	                 67.40	                62702
Arunachal Pradesh	            1.38	                 66.95	                4047
Assam	                        31.17	                 73.18	                71042
Bihar	                        119.24	                 63.82	                84236

OUTPUT:

Population(Millions) Literacy Rate(%) No of Schools

State

Andaman and Nicobar Islands 0.38 86.27 414

Andhra Pradesh 84.67 67.40 62702

Arunachal Pradesh 1.38 66.95 4047

Assam 31.17 73.18 71042

Bihar 119.24 63.82 84236

Note that the original ‘data’ object is changed when inplace=True

We can use reset_index() to reset the index to its default values.

print(df.reset_index().head())

1	print(df.reset_index().head())

OUTPUT:
     State	                        Population(Millions)    Literacy Rate(%)    No of Schools  
0    Andaman and Nicobar Islands	0.38                    86.27               414
1    Andhra Pradesh	                84.67                   67.40               62702
2    Arunachal Pradesh	            1.38                    66.95               4047
3    Assam	                        31.17                   73.18               71042
4    Bihar	                        119.24                  63.82               84236

OUTPUT:

State Population(Millions) Literacy Rate(%) No of Schools

0 Andaman and Nicobar Islands 0.38 86.27 414

1 Andhra Pradesh 84.67 67.40 62702

2 Arunachal Pradesh 1.38 66.95 4047

3 Assam 31.17 73.18 71042

4 Bihar 119.24 63.82 84236

Note that we didn’t set inplace = True. So, the operation won’t affect the original data object.

Subsetting rows and columns:

If we want to access the data in individual columns, we can use the [ ] operator.

print(df['Population(Millions)'].head())

1	print(df['Population(Millions)'].head())

OUTPUT:
State
Andaman and Nicobar Islands      0.38
Andhra Pradesh                  84.67
Arunachal Pradesh                1.38
Assam                           31.17
Bihar                          119.24
Name: Population(Millions), dtype: float64

OUTPUT:

State

Andaman and Nicobar Islands 0.38

Andhra Pradesh 84.67

Arunachal Pradesh 1.38

Assam 31.17

Bihar 119.24

Name: Population(Millions), dtype: float64

To specify multiple columns, we need to pass in a Python list

print(df[['Population(Millions)', 'No of Schools']].head())

1	print(df[['Population(Millions)', 'No of Schools']].head())

OUTPUT:
                                  Population(Millions)  No of Schools
                      State                                              
Andaman and Nicobar Islands                  0.38            414
Andhra Pradesh                              84.67          62702
Arunachal Pradesh                            1.38           4047
Assam                                       31.17          71042
Bihar                                      119.24          84236

OUTPUT:

Population(Millions) No of Schools

State

Andaman and Nicobar Islands 0.38 414

Andhra Pradesh 84.67 62702

Arunachal Pradesh 1.38 4047

Assam 31.17 71042

Bihar 119.24 84236

We can subset rows in various ways, for label based indexing we can use loc, and for positional based indexing we can use iloc.

print(df.loc['Tamil Nadu'])

1	print(df.loc['Tamil Nadu'])

OUTPUT:
Population(Millions)       79.10
Literacy Rate(%)           80.33
No of Schools           57583.00
Name: Tamil Nadu, dtype: float64

OUTPUT:

Population(Millions) 79.10

Literacy Rate(%) 80.33

No of Schools 57583.00

Name: Tamil Nadu, dtype: float64

print(df.iloc[30])

1	print(df.iloc[30])

Population(Millions)       79.10
Literacy Rate(%)           80.33
No of Schools           57583.00
Name: Tamil Nadu, dtype: float64

Population(Millions) 79.10

Literacy Rate(%) 80.33

No of Schools 57583.00

Name: Tamil Nadu, dtype: float64

If you want to get the numerical location of the index, you can use the index.get_loc() method.

print(df.index.get_loc('Haryana'))

1	print(df.index.get_loc('Haryana'))

OUTPUT:
12

1 2	OUTPUT: 12

We can also get the value of a particular cell. For example, let’s find out how many schools are there in Maharashtra.

print(df.loc['Maharashtra']['No of Schools'])

1	print(df.loc['Maharashtra']['No of Schools'])

OUTPUT:
107624.0

1 2	OUTPUT: 107624.0

Alternatively, we can also use iloc if we need to use the numerical position

print(df.iloc[20][2])

1	print(df.iloc[20][2])

OUTPUT: 
107624.0

1 2	OUTPUT: 107624.0

Renaming column names:

We can use the .rename() method to rename the column names.

We’ll remove the parenthesis from the columns population and literacy rate and replace white space in the school column with an underscore.

df.rename(columns={'Population(Millions)':'Population', \
'Literacy Rate(%)':'LiteracyRate', \
'No of Schools':'No_of_Schools'}, inplace=True)
print(df.columns)

df.rename(columns={'Population(Millions)':'Population', \

'Literacy Rate(%)':'LiteracyRate', \

'No of Schools':'No_of_Schools'}, inplace=True)

print(df.columns)

Index(['Population', 'LiteracyRate', 'No_of_Schools'], dtype='object')

1	Index(['Population', 'LiteracyRate', 'No_of_Schools'], dtype='object')

We can also rename indexes in the same way.

df.rename(index={'Gujrat':'Gujarat'}, inplace=True)
print(df.index)

1 2	df.rename(index={'Gujrat':'Gujarat'}, inplace=True) print(df.index)

Index(['Andaman and Nicobar Islands', 'Andhra Pradesh', 'Arunachal Pradesh',
       'Assam', 'Bihar', 'Chandigarh', 'Chhattisgarh',
       'Dadra and Nagar Haveli', 'Daman and Diu', 'Delhi', 'Goa', 'Gujarat',
       'Haryana', 'Himachal Pradesh', 'Jammu and Kashmir', 'Jharkhand',
       'Karnataka', 'Kerala', 'Lakshadweep', 'Madhya Pradesh', 'Maharashtra',
       'Manipur', 'Meghalaya', 'Mizoram', 'Nagaland', 'Orissa', 'Puducherry',
       'Punjab', 'Rajasthan', 'Sikkim', 'Tamil Nadu', 'Telangana', 'Tripura',
       'Uttar Pradesh', 'Uttarakhand', 'West Bengal'],
      dtype='object', name='State')

Index(['Andaman and Nicobar Islands', 'Andhra Pradesh', 'Arunachal Pradesh',

'Assam', 'Bihar', 'Chandigarh', 'Chhattisgarh',

'Dadra and Nagar Haveli', 'Daman and Diu', 'Delhi', 'Goa', 'Gujarat',

'Haryana', 'Himachal Pradesh', 'Jammu and Kashmir', 'Jharkhand',

'Karnataka', 'Kerala', 'Lakshadweep', 'Madhya Pradesh', 'Maharashtra',

'Manipur', 'Meghalaya', 'Mizoram', 'Nagaland', 'Orissa', 'Puducherry',

'Punjab', 'Rajasthan', 'Sikkim', 'Tamil Nadu', 'Telangana', 'Tripura',

'Uttar Pradesh', 'Uttarakhand', 'West Bengal'],

dtype='object', name='State')

Iterating through rows and columns:

By using the iterrows() method we can iterate through the rows

for index, series in df.head().iterrows():
    print(index, '\n', series, '\n')

1 2	for index, series in df.head().iterrows(): print(index, '\n', series, '\n')

OUTPUT:
Andaman and Nicobar Islands 
Population         0.38
LiteracyRate      86.27
No_of_Schools    414.00
Name: Andaman and Nicobar Islands, dtype: float64 

Andhra Pradesh 
Population          84.67
LiteracyRate        67.40
No_of_Schools    62702.00
Name: Andhra Pradesh, dtype: float64 

Arunachal Pradesh 
Population          1.38
LiteracyRate       66.95
No_of_Schools    4047.00
Name: Arunachal Pradesh, dtype: float64 

Assam 
Population          31.17
LiteracyRate        73.18
No_of_Schools    71042.00
Name: Assam, dtype: float64 

Bihar 
Population         119.24
LiteracyRate        63.82
No_of_Schools    84236.00
Name: Bihar, dtype: float64

OUTPUT:

Andaman and Nicobar Islands

Population 0.38

LiteracyRate 86.27

No_of_Schools 414.00

Name: Andaman and Nicobar Islands, dtype: float64

Andhra Pradesh

Population 84.67

LiteracyRate 67.40

No_of_Schools 62702.00

Name: Andhra Pradesh, dtype: float64

Arunachal Pradesh

Population 1.38

LiteracyRate 66.95

No_of_Schools 4047.00

Name: Arunachal Pradesh, dtype: float64

Assam

Population 31.17

LiteracyRate 73.18

No_of_Schools 71042.00

Name: Assam, dtype: float64

Bihar

Population 119.24

LiteracyRate 63.82

No_of_Schools 84236.00

Name: Bihar, dtype: float64

Another way to iterate rows is to use itertuple(). It returns a named tuple.

for row in df.head().itertuples():
    print(row)

1 2	for row in df.head().itertuples(): print(row)

OUTPUT:
Pandas(Index='Andaman and Nicobar Islands', Population=0.38, LiteracyRate=86.27, No_of_Schools=414)
Pandas(Index='Andhra Pradesh', Population=84.67, LiteracyRate=67.4, No_of_Schools=62702)
Pandas(Index='Arunachal Pradesh', Population=1.38, LiteracyRate=66.95, No_of_Schools=4047)
Pandas(Index='Assam', Population=31.17, LiteracyRate=73.18, No_of_Schools=71042)
Pandas(Index='Bihar', Population=119.24, LiteracyRate=63.82, No_of_Schools=84236)

OUTPUT:

Pandas(Index='Andaman and Nicobar Islands', Population=0.38, LiteracyRate=86.27, No_of_Schools=414)

Pandas(Index='Andhra Pradesh', Population=84.67, LiteracyRate=67.4, No_of_Schools=62702)

Pandas(Index='Arunachal Pradesh', Population=1.38, LiteracyRate=66.95, No_of_Schools=4047)

Pandas(Index='Assam', Population=31.17, LiteracyRate=73.18, No_of_Schools=71042)

Pandas(Index='Bihar', Population=119.24, LiteracyRate=63.82, No_of_Schools=84236)

To iterate over columns, we can use iteritems(). It iterates over the DataFrame columns, returning a tuple with the column name and the content as a Series.

for col, series in df.head().iteritems():
    print(col, series, "\n")

1 2	for col, series in df.head().iteritems(): print(col, series, "\n")

OUTPUT:
Population State
Andaman and Nicobar Islands      0.38
Andhra Pradesh                  84.67
Arunachal Pradesh                1.38
Assam                           31.17
Bihar                          119.24
Name: Population, dtype: float64 

LiteracyRate State
Andaman and Nicobar Islands    86.27
Andhra Pradesh                 67.40
Arunachal Pradesh              66.95
Assam                          73.18
Bihar                          63.82
Name: LiteracyRate, dtype: float64 

No_of_Schools State
Andaman and Nicobar Islands      414
Andhra Pradesh                 62702
Arunachal Pradesh               4047
Assam                          71042
Bihar                          84236
Name: No_of_Schools, dtype: int64

OUTPUT:

Population State

Andaman and Nicobar Islands 0.38

Andhra Pradesh 84.67

Arunachal Pradesh 1.38

Assam 31.17

Bihar 119.24

Name: Population, dtype: float64

LiteracyRate State

Andaman and Nicobar Islands 86.27

Andhra Pradesh 67.40

Arunachal Pradesh 66.95

Assam 73.18

Bihar 63.82

Name: LiteracyRate, dtype: float64

No_of_Schools State

Andaman and Nicobar Islands 414

Andhra Pradesh 62702

Arunachal Pradesh 4047

Assam 71042

Bihar 84236

Name: No_of_Schools, dtype: int64

Creating and Deleting a column:

Let’s see how to add a new column to a pandas DataFrame.

We can create a new column by using the [ ] operator.

We’ll create a new column pop_rank which will contain the rank of the population.

df['Pop_Rank'] = df['Population'].rank(ascending=False)
print(df.head())

1 2	df['Pop_Rank'] = df['Population'].rank(ascending=False) print(df.head())

OUTPUT:
                               Population  LiteracyRate  No_of_Schools  Pop_Rank
State                                                                         
Andaman and Nicobar Islands        0.38         86.27            414      33.0
Andhra Pradesh                    84.67         67.40          62702       5.0
Arunachal Pradesh                  1.38         66.95           4047      28.0
Assam                             31.17         73.18          71042      15.0
Bihar                            119.24         63.82          84236       3.0

OUTPUT:

Population LiteracyRate No_of_Schools Pop_Rank

State

Andaman and Nicobar Islands 0.38 86.27 414 33.0

Andhra Pradesh 84.67 67.40 62702 5.0

Arunachal Pradesh 1.38 66.95 4047 28.0

Assam 31.17 73.18 71042 15.0

Bihar 119.24 63.82 84236 3.0

We can also use the assign() method to add a new column to a pandas DataFrame. It returns a new object with all original columns in addition to new ones.

df = df.assign(lit_rank=df['LiteracyRate'].rank(ascending=False))
print(df.head())

1 2	df = df.assign(lit_rank=df['LiteracyRate'].rank(ascending=False)) print(df.head())

OUTPUT:
                               Population  LiteracyRate  No_of_Schools  \
State                                                                  
Andaman and Nicobar Islands        0.38         86.27            414   
Andhra Pradesh                    84.67         67.40          62702   
Arunachal Pradesh                  1.38         66.95           4047   
Assam                             31.17         73.18          71042   
Bihar                            119.24         63.82          84236   

                             Pop_Rank  lit_rank  
State                                            
Andaman and Nicobar Islands      33.0      10.0  
Andhra Pradesh                    5.0      32.0  
Arunachal Pradesh                28.0      34.0  
Assam                            15.0      26.0  
Bihar                             3.0      36.0

OUTPUT:

Population LiteracyRate No_of_Schools \

State

Andaman and Nicobar Islands 0.38 86.27 414

Andhra Pradesh 84.67 67.40 62702

Arunachal Pradesh 1.38 66.95 4047

Assam 31.17 73.18 71042

Bihar 119.24 63.82 84236

Pop_Rank lit_rank

State

Andaman and Nicobar Islands 33.0 10.0

Andhra Pradesh 5.0 32.0

Arunachal Pradesh 28.0 34.0

Assam 15.0 26.0

Bihar 3.0 36.0

We can remove a column by using the drop() method. The following code drops the lit_rank column.

df.drop('lit_rank', axis=1, inplace=True)
print(df.head())

1 2	df.drop('lit_rank', axis=1, inplace=True) print(df.head())

OUTPUT:
                               Population  LiteracyRate  No_of_Schools  Pop_Rank
State                                                                         
Andaman and Nicobar Islands        0.38         86.27            414      33.0
Andhra Pradesh                    84.67         67.40          62702       5.0
Arunachal Pradesh                  1.38         66.95           4047      28.0
Assam                             31.17         73.18          71042      15.0
Bihar                            119.24         63.82          84236       3.0

OUTPUT:

Population LiteracyRate No_of_Schools Pop_Rank

State

Andaman and Nicobar Islands 0.38 86.27 414 33.0

Andhra Pradesh 84.67 67.40 62702 5.0

Arunachal Pradesh 1.38 66.95 4047 28.0

Assam 31.17 73.18 71042 15.0

Bihar 119.24 63.82 84236 3.0

Rearrange Columns:

We can change the order of the columns in a pandas DataFrame easily. We just need to pass the list of column names in the desired order.

df = df[['Population', 'Pop_Rank', 'LiteracyRate', 'No_of_Schools']]
print(df.head())

1 2	df = df[['Population', 'Pop_Rank', 'LiteracyRate', 'No_of_Schools']] print(df.head())

OUTPUT:
                               Population  Pop_Rank  LiteracyRate  No_of_Schools
State                                                                         
Andaman and Nicobar Islands        0.38      33.0         86.27            414
Andhra Pradesh                    84.67       5.0         67.40          62702
Arunachal Pradesh                  1.38      28.0         66.95           4047
Assam                             31.17      15.0         73.18          71042
Bihar                            119.24       3.0         63.82          84236

OUTPUT:

Population Pop_Rank LiteracyRate No_of_Schools

State

Andaman and Nicobar Islands 0.38 33.0 86.27 414

Andhra Pradesh 84.67 5.0 67.40 62702

Arunachal Pradesh 1.38 28.0 66.95 4047

Assam 31.17 15.0 73.18 71042

Bihar 119.24 3.0 63.82 84236

Save DataFrame as CSV:

Once we are done, we can save the DataFrame as a CSV file. To demonstrate saving data to CSV file, we’ll save the df object to a new file named data_new.csv

df.to_csv('data_new.csv')

1	df.to_csv('data_new.csv')

SUMMARY:

In this article, we learned about some of the basics of Pandas, the most-popular data analysis library for Python.

There are two main data structures in pandas: Series and DataFrame.

Series: A pandas series is a one-dimensional array that contains a sequence of values. It’s similar to a Numpy array.

DataFrame: A DataFrame is a two-dimensional data structure made up of rows and columns.

With a series of examples, we also learned some of the standard operations of pandas for data analysis.

Posted in pandas

Love What you Read. Subscribe to our Newsletter.

Stay up to date! We’ll send the content straight to your inbox, once a week. We promise not to spam you.