Data Sets in Python

There are various formats for a dataset, .csv, .json, .xlsx, etc. The dataset can be stored in different places, on your local machine, or sometimes online. Many of the Python libraries are bundled with data set(s). One can install and load/import data sets in Python. For example, the vega_datasets is a Python library that contains some popular data sets.

# install vega datasets
pip install vega_datasets

After successful installation of the vega_datasets library, one needs to import and load the data.

from vega_datasets import data
iris = data.iris()

Now the datasets bundled in vega_datasets are locally available and one can readily access them without an internet connection. One can see the list of available data sets in the vega_datasets library.

vega_datasets in Python

Getting Data Set Information in Python

Since the iris data set is assigned to the variable iris. Now one can perform different computations on this variable.

# unique observations in iris

# identifying missing observations in the data set
Missing Observations in a Data
# checking the data types of variables in the data set

Computing Statistics in Python

After checking some information and cleaning the data, one can perform different statistics on the data. For example,

# Descriptive Statistics

If a variable is not required for further computations, one can drop the variable and may create a new data set. For example, from the iris data set, if a “species” variable is not required, one can omit the variable and may create a new data set.

# data = iris.drop(['species'], axis = 1)

The argument axis=1 refers to the columns of a data frame or a series and axis=0 refers to the rows of a data frame or a series.

Reading Online Data Sets in Python

Note that one can also read the data from a URL. For example,

URL = ""
df = pd.read_csv(URL, header = None)

For any data sets, one may update the column (variable) names. For the data stored in the URL, the column names are 0, 1, 2, …, 25. Therefore, one can name the columns with some strings, like

headers = ["symboling", "normalized-losses", "make", "fuel-type", "aspiration", "num-of-doors", "body-style", "drive-wheels", "engine-location", "wheel-base", "length", "width", "height", "curb-weight", "engine-type", "num-of-cylinders", "engine-size", "fuel-system", "bore", "stroke", "compression-ratio", "horsepower", "peak-rpm", "city-mpg", "highway-mpg", "price"]
df.columns = headers
Data sets in Python

Exporting Data Sets in Python

The data sets in Python can also be imported to diffeent file format. For example

# export as csf file
df.to_csv("data.csv", index = False)

# export as excel file
df.to_excel("data.xlsx", index = False)

Instead of importing data set, one can also generate his/her own data set.

before = stats.norm.rvs(scale = 30, loc = 250, size = 100)

after  = before + stats.norm.rvs(scale = 5, loc = -1.25, size = 100)

df = pd.DataFrame(
                {"Weight_before": before,
                   "Weight_after": after,
                   "diff": after-before}
Generated Data set in Python