To perform EDA on the given data set.
The primary aim with exploratory analysis is to examine the data for distribution, outliers and anomalies to direct specific testing of your hypothesis.
Import the required packages(pandas,numpy,seaborn).
Read and Load the Dataset
Remove the null values from the data and remove the outliers.
Remove the non numerical data columns using drop() method.
returns object containing counts of unique values using (value_counts()).
Plot the counts in the form of Histogram or Bar Graph.
find the pairwise correlation of all columns in the dataframe(.corr()).
Save the final data set into the file.
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
df= pd.read_csv('titanic_dataset.csv')
df.head()
df.info()
df.isnull().sum()
df['Cabin']=df['Cabin'].fillna(df['Cabin'].mode()[0])
df['Age']=df['Age'].fillna(df['Age'].mean())
df['Embarked']=df['Embarked'].fillna(df['Embarked'].mode()[0])
df.isnull().sum()
df.boxplot()
cols = ['Age', 'Fare','SibSp','Parch','Fare']
Q1 = df[cols].quantile(0.25)
Q3 = df[cols].quantile(0.75)
IQR = Q3 - Q1
df = df[~((df[cols] < (Q1 - 1.5 * IQR)) |(df[cols] > (Q3 + 1.5 * IQR))).any(axis=1)]
df.boxplot()
survived_count = df.groupby('Survived')['Survived'].count()
survived_count
plt.title('Grouped by survival')
sns.countplot(x="Survived",data=df)
df["Pclass"].value_counts()
plt.title('Grouped by pclass')
sns.countplot(x="Pclass",data=df)
df["Sex"].value_counts()
plt.title('Grouped by gender')
sns.countplot(x="Sex",data=df)
df["Embarked"].value_counts()
plt.title('Grouped by embarkation')
sns.countplot(x="Embarked",data=df)
sns.displot(df["Fare"])
plt.title('Survived female and male')
sns.countplot(x="Sex",hue="Survived",data=df)
sns.countplot(x="Pclass",hue="Survived",data=df)
sns.countplot(x="Sex",hue="Survived",data=df)
sns.displot(df[df["Survived"]==0]["Age"])
sns.displot(df[df["Survived"]==1]["Age"])
pd.crosstab(df["Pclass"],df["Survived"])
pd.crosstab(df["Sex"],df["Survived"])
df.corr()
sns.heatmap(df.corr(),annot=True)
The data has been cleaned, outlier has been removed and the EDA on the given data has been performed.