Pandas Profiling Report

Dataset statistics

Number of variables	10
Number of observations	50009
Missing cells	9
Missing cells (%)	< 0.1%
Duplicate rows	1
Duplicate rows (%)	< 0.1%
Total size in memory	3.8 MiB
Average record size in memory	80.0 B

Variable types

Numeric	1
Categorical	9

Alerts

Dataset has 1 (< 0.1%) duplicate rows	Duplicates
`respondent` has a high cardinality: 77 distinct values	High cardinality
`respondent-name` has a high cardinality: 77 distinct values	High cardinality
`value` has a high cardinality: 19112 distinct values	High cardinality
`Unnamed: 0` is highly correlated with `period`	High correlation
`period` is highly correlated with `Unnamed: 0` and 7 other fields	High correlation
`respondent` is highly correlated with `period` and 6 other fields	High correlation
`respondent-name` is highly correlated with `period` and 6 other fields	High correlation
`fueltype` is highly correlated with `period` and 6 other fields	High correlation
`type-name` is highly correlated with `period` and 6 other fields	High correlation
`timezone` is highly correlated with `period` and 6 other fields	High correlation
`timezone-description` is highly correlated with `period` and 6 other fields	High correlation
`value-units` is highly correlated with `period` and 6 other fields	High correlation
`Unnamed: 0` is uniformly distributed	Uniform

Reproduction

Analysis started	2022-11-17 22:35:52.894493
Analysis finished	2022-11-17 22:35:57.080879
Duration	4.19 seconds
Software version	pandas-profiling v3.4.0
Download configuration	config.json

Unnamed: 0
Real number (ℝ_≥0)

HIGH CORRELATION
UNIFORM

Distinct	5000
Distinct (%)	10.0%
Missing	9
Missing (%)	< 0.1%
Infinite	0
Infinite (%)	0.0%
Mean	2499.5

Minimum	0
Maximum	4999
Zeros	10
Zeros (%)	< 0.1%
Negative	0
Negative (%)	0.0%
Memory size	390.8 KiB

Quantile statistics

Minimum	0
5-th percentile	249.95
Q1	1249.75
median	2499.5
Q3	3749.25
95-th percentile	4749.05
Maximum	4999
Range	4999
Interquartile range (IQR)	2499.5

Descriptive statistics

Standard deviation	1443.390078
Coefficient of variation (CV)	0.5774715255
Kurtosis	-1.200000095
Mean	2499.5
Median Absolute Deviation (MAD)	1250
Skewness	0
Sum	124975000
Variance	2083374.917
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	10	< 0.1%
3331	10	< 0.1%
3338	10	< 0.1%
3337	10	< 0.1%
3336	10	< 0.1%
3335	10	< 0.1%
3334	10	< 0.1%
3333	10	< 0.1%
3332	10	< 0.1%
3330	10	< 0.1%
Other values (4990)	49900	99.8%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
0	10	< 0.1%
1	10	< 0.1%
2	10	< 0.1%
3	10	< 0.1%
4	10	< 0.1%
5	10	< 0.1%
6	10	< 0.1%
7	10	< 0.1%
8	10	< 0.1%
9	10	< 0.1%

Value	Count	Frequency (%)
4999	10	< 0.1%
4998	10	< 0.1%
4997	10	< 0.1%
4996	10	< 0.1%
4995	10	< 0.1%
4994	10	< 0.1%
4993	10	< 0.1%
4992	10	< 0.1%
4991	10	< 0.1%
4990	10	< 0.1%

period
Categorical

HIGH CORRELATION

Distinct	28
Distinct (%)	0.1%
Missing	0
Missing (%)	0.0%
Memory size	390.8 KiB

2022-10-31	1945
2022-10-22	1945
2022-10-30	1944
2022-10-21	1944
2022-10-23	1941
Other values (23)	40290

Length

Max length	10
Median length	10
Mean length	9.99928013
Min length	6

Characters and Unicode

Total characters	500054
Distinct characters	17
Distinct categories	3 ?
Distinct scripts	2 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	2022-11-13
2nd row	2022-11-13
3rd row	2022-11-13
4th row	2022-11-13
5th row	2022-11-13

Common Values

Value	Count	Frequency (%)
2022-10-31	1945	3.9%
2022-10-22	1945	3.9%
2022-10-30	1944	3.9%
2022-10-21	1944	3.9%
2022-10-23	1941	3.9%
2022-10-19	1940	3.9%
2022-11-10	1940	3.9%
2022-11-11	1940	3.9%
2022-10-24	1940	3.9%
2022-10-25	1940	3.9%
Other values (18)	30590	61.2%

Length

Histogram of lengths of the category

Value	Count	Frequency (%)
2022-10-31	1945	3.9%
2022-10-22	1945	3.9%
2022-10-30	1944	3.9%
2022-10-21	1944	3.9%
2022-10-23	1941	3.9%
2022-10-29	1940	3.9%
2022-11-08	1940	3.9%
2022-11-07	1940	3.9%
2022-11-05	1940	3.9%
2022-11-04	1940	3.9%
Other values (18)	30590	61.2%

Most occurring characters

Value	Count	Frequency (%)
2	175165	35.0%
-	100000	20.0%
0	99065	19.8%
1	91229	18.2%
3	8779	1.8%
9	5797	1.2%
8	4447	0.9%
4	3880	0.8%
5	3880	0.8%
7	3880	0.8%
Other values (7)	3932	0.8%

Most occurring categories

Value	Count	Frequency (%)
Decimal Number	400000	80.0%
Dash Punctuation	100000	20.0%
Lowercase Letter	54	< 0.1%

Most frequent character per category

Decimal Number

Value	Count	Frequency (%)
2	175165	43.8%
0	99065	24.8%
1	91229	22.8%
3	8779	2.2%
9	5797	1.4%
8	4447	1.1%
4	3880	1.0%
5	3880	1.0%
7	3880	1.0%
6	3878	1.0%

Lowercase Letter

Value	Count	Frequency (%)
p	9	16.7%
e	9	16.7%
r	9	16.7%
i	9	16.7%
o	9	16.7%
d	9	16.7%

Dash Punctuation

Value	Count	Frequency (%)
-	100000	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Common	500000	> 99.9%
Latin	54	< 0.1%

Most frequent character per script

Common

Value	Count	Frequency (%)
2	175165	35.0%
-	100000	20.0%
0	99065	19.8%
1	91229	18.2%
3	8779	1.8%
9	5797	1.2%
8	4447	0.9%
4	3880	0.8%
5	3880	0.8%
7	3880	0.8%

Latin

Value	Count	Frequency (%)
p	9	16.7%
e	9	16.7%
r	9	16.7%
i	9	16.7%
o	9	16.7%
d	9	16.7%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	500054	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
2	175165	35.0%
-	100000	20.0%
0	99065	19.8%
1	91229	18.2%
3	8779	1.8%
9	5797	1.2%
8	4447	0.9%
4	3880	0.8%
5	3880	0.8%
7	3880	0.8%
Other values (7)	3932	0.8%

respondent
Categorical

HIGH CARDINALITY
HIGH CORRELATION

Distinct	77
Distinct (%)	0.2%
Missing	0
Missing (%)	0.0%
Memory size	390.8 KiB

ISNE	1055
NY	1050
NE	1048
SOCO	1045
US48	1041
Other values (72)	44770

Length

Max length	10
Median length	4
Mean length	3.443020256
Min length	2

Characters and Unicode

Total characters	172182
Distinct characters	35
Distinct categories	3 ?
Distinct scripts	2 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	PACE
2nd row	PACE
3rd row	IID
4th row	IPCO
5th row	IID

Common Values

Value	Count	Frequency (%)
ISNE	1055	2.1%
NY	1050	2.1%
NE	1048	2.1%
SOCO	1045	2.1%
US48	1041	2.1%
SWPP	1039	2.1%
TEN	1039	2.1%
SW	1038	2.1%
MIDA	1037	2.1%
NEVP	1037	2.1%
Other values (67)	39580	79.1%

Length

Histogram of lengths of the category

Value	Count	Frequency (%)
isne	1055	2.1%
ny	1050	2.1%
ne	1048	2.1%
soco	1045	2.1%
us48	1041	2.1%
swpp	1039	2.1%
ten	1039	2.1%
sw	1038	2.1%
tva	1037	2.1%
nevp	1037	2.1%
Other values (67)	39580	79.1%

Most occurring characters

Value	Count	Frequency (%)
P	19309	11.2%
C	17280	10.0%
E	16601	9.6%
S	15346	8.9%
A	14152	8.2%
N	10870	6.3%
I	9880	5.7%
W	8691	5.0%
T	7588	4.4%
M	6901	4.0%
Other values (25)	45564	26.5%

Most occurring categories

Value	Count	Frequency (%)
Uppercase Letter	170010	98.7%
Decimal Number	2082	1.2%
Lowercase Letter	90	0.1%

Most frequent character per category

Uppercase Letter

Value	Count	Frequency (%)
P	19309	11.4%
C	17280	10.2%
E	16601	9.8%
S	15346	9.0%
A	14152	8.3%
N	10870	6.4%
I	9880	5.8%
W	8691	5.1%
T	7588	4.5%
M	6901	4.1%
Other values (15)	43392	25.5%

Lowercase Letter

Value	Count	Frequency (%)
e	18	20.0%
n	18	20.0%
r	9	10.0%
s	9	10.0%
p	9	10.0%
o	9	10.0%
d	9	10.0%
t	9	10.0%

Decimal Number

Value	Count	Frequency (%)
8	1041	50.0%
4	1041	50.0%

Most occurring scripts

Value	Count	Frequency (%)
Latin	170100	98.8%
Common	2082	1.2%

Most frequent character per script

Latin

Value	Count	Frequency (%)
P	19309	11.4%
C	17280	10.2%
E	16601	9.8%
S	15346	9.0%
A	14152	8.3%
N	10870	6.4%
I	9880	5.8%
W	8691	5.1%
T	7588	4.5%
M	6901	4.1%
Other values (23)	43482	25.6%

Common

Value	Count	Frequency (%)
8	1041	50.0%
4	1041	50.0%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	172182	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
P	19309	11.2%
C	17280	10.0%
E	16601	9.6%
S	15346	8.9%
A	14152	8.2%
N	10870	6.3%
I	9880	5.7%
W	8691	5.0%
T	7588	4.4%
M	6901	4.0%
Other values (25)	45564	26.5%

respondent-name
Categorical

HIGH CARDINALITY
HIGH CORRELATION

Distinct	77
Distinct (%)	0.2%
Missing	0
Missing (%)	0.0%
Memory size	390.8 KiB

ISO New England	1055
New York	1050
New England	1048
Southern Company Services, Inc. - Trans	1045
United States Lower 48	1041
Other values (72)	44770

Length

Max length	66
Median length	39
Mean length	24.83426983
Min length	3

Characters and Unicode

Total characters	1241937
Distinct characters	58
Distinct categories	6 ?
Distinct scripts	2 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	PacifiCorp East
2nd row	PacifiCorp East
3rd row	Imperial Irrigation District
4th row	Idaho Power Company
5th row	Imperial Irrigation District

Common Values

Value	Count	Frequency (%)
ISO New England	1055	2.1%
New York	1050	2.1%
New England	1048	2.1%
Southern Company Services, Inc. - Trans	1045	2.1%
United States Lower 48	1041	2.1%
Southwest Power Pool	1039	2.1%
Tennessee	1039	2.1%
Southwest	1038	2.1%
Mid-Atlantic	1037	2.1%
Nevada Power Company	1037	2.1%
Other values (67)	39580	79.1%

Length

Histogram of lengths of the category

Value	Count	Frequency (%)
power	9222	5.4%
company	8145	4.8%
inc	5764	3.4%
energy	5458	3.2%
new	5081	3.0%
of	4810	2.8%
electric	4412	2.6%
public	3852	2.2%
service	3334	1.9%
duke	3134	1.8%
Other values (129)	118059	68.9%

Most occurring characters

Value	Count	Frequency (%)
	121262	9.8%
e	113206	9.1%
n	88026	7.1%
o	87515	7.0%
r	82035	6.6%
t	78090	6.3%
i	77031	6.2%
a	71405	5.7%
l	43626	3.5%
s	40955	3.3%
Other values (48)	438786	35.3%

Most occurring categories

Value	Count	Frequency (%)
Lowercase Letter	924372	74.4%
Uppercase Letter	173763	14.0%
Space Separator	121262	9.8%
Other Punctuation	16439	1.3%
Dash Punctuation	3504	0.3%
Decimal Number	2597	0.2%

Most frequent character per category

Lowercase Letter

Value	Count	Frequency (%)
e	113206	12.2%
n	88026	9.5%
o	87515	9.5%
r	82035	8.9%
t	78090	8.4%
i	77031	8.3%
a	71405	7.7%
l	43626	4.7%
s	40955	4.4%
c	38437	4.2%
Other values (16)	204046	22.1%

Uppercase Letter

Value	Count	Frequency (%)
C	25006	14.4%
P	21879	12.6%
S	18315	10.5%
E	14832	8.5%
I	13711	7.9%
A	12179	7.0%
N	9180	5.3%
L	8607	5.0%
D	7825	4.5%
T	6842	3.9%
Other values (13)	35387	20.4%

Decimal Number

Value	Count	Frequency (%)
8	1041	40.1%
4	1041	40.1%
1	387	14.9%
2	128	4.9%

Other Punctuation

Value	Count	Frequency (%)
,	8860	53.9%
.	6929	42.1%
&	650	4.0%

Space Separator

Value	Count	Frequency (%)
	121262	100.0%

Dash Punctuation

Value	Count	Frequency (%)
-	3504	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Latin	1098135	88.4%
Common	143802	11.6%

Most frequent character per script

Latin

Value	Count	Frequency (%)
e	113206	10.3%
n	88026	8.0%
o	87515	8.0%
r	82035	7.5%
t	78090	7.1%
i	77031	7.0%
a	71405	6.5%
l	43626	4.0%
s	40955	3.7%
c	38437	3.5%
Other values (39)	377809	34.4%

Common

Value	Count	Frequency (%)
	121262	84.3%
,	8860	6.2%
.	6929	4.8%
-	3504	2.4%
8	1041	0.7%
4	1041	0.7%
&	650	0.5%
1	387	0.3%
2	128	0.1%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	1241937	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
	121262	9.8%
e	113206	9.1%
n	88026	7.1%
o	87515	7.0%
r	82035	6.6%
t	78090	6.3%
i	77031	6.2%
a	71405	5.7%
l	43626	3.5%
s	40955	3.3%
Other values (48)	438786	35.3%

fueltype
Categorical

HIGH CORRELATION

Distinct	9
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	390.8 KiB

NG	8356
WAT	7560
SUN	7356
OTH	6452
COL	6318
Other values (4)	13967

Length

Max length	8
Median length	3
Mean length	2.833809914
Min length	2

Characters and Unicode

Total characters	141716
Distinct characters	20
Distinct categories	2 ?
Distinct scripts	1 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	NG
2nd row	NG
3rd row	NG
4th row	WAT
5th row	NG

Common Values

Value	Count	Frequency (%)
NG	8356	16.7%
WAT	7560	15.1%
SUN	7356	14.7%
OTH	6452	12.9%
COL	6318	12.6%
WND	5665	11.3%
OIL	4159	8.3%
NUC	4134	8.3%
fueltype	9	< 0.1%

Length

Histogram of lengths of the category

Category Frequency Plot

Value	Count	Frequency (%)
ng	8356	16.7%
wat	7560	15.1%
sun	7356	14.7%
oth	6452	12.9%
col	6318	12.6%
wnd	5665	11.3%
oil	4159	8.3%
nuc	4134	8.3%
fueltype	9	< 0.1%

Most occurring characters

Value	Count	Frequency (%)
N	25511	18.0%
O	16929	11.9%
T	14012	9.9%
W	13225	9.3%
U	11490	8.1%
L	10477	7.4%
C	10452	7.4%
G	8356	5.9%
A	7560	5.3%
S	7356	5.2%
Other values (10)	16348	11.5%

Most occurring categories

Value	Count	Frequency (%)
Uppercase Letter	141644	99.9%
Lowercase Letter	72	0.1%

Most frequent character per category

Uppercase Letter

Value	Count	Frequency (%)
N	25511	18.0%
O	16929	12.0%
T	14012	9.9%
W	13225	9.3%
U	11490	8.1%
L	10477	7.4%
C	10452	7.4%
G	8356	5.9%
A	7560	5.3%
S	7356	5.2%
Other values (3)	16276	11.5%

Lowercase Letter

Value	Count	Frequency (%)
e	18	25.0%
f	9	12.5%
u	9	12.5%
l	9	12.5%
t	9	12.5%
y	9	12.5%
p	9	12.5%

Most occurring scripts

Value	Count	Frequency (%)
Latin	141716	100.0%

Most frequent character per script

Latin

Value	Count	Frequency (%)
N	25511	18.0%
O	16929	11.9%
T	14012	9.9%
W	13225	9.3%
U	11490	8.1%
L	10477	7.4%
C	10452	7.4%
G	8356	5.9%
A	7560	5.3%
S	7356	5.2%
Other values (10)	16348	11.5%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	141716	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
N	25511	18.0%
O	16929	11.9%
T	14012	9.9%
W	13225	9.3%
U	11490	8.1%
L	10477	7.4%
C	10452	7.4%
G	8356	5.9%
A	7560	5.3%
S	7356	5.2%
Other values (10)	16348	11.5%

type-name
Categorical

HIGH CORRELATION

Distinct	9
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	390.8 KiB

Natural gas	8356
Hydro	7560
Solar	7356
Other	6452
Coal	6318
Other values (4)	13967

Length

Max length	11
Median length	9
Mean length	6.261632906
Min length	4

Characters and Unicode

Total characters	313138
Distinct characters	26
Distinct categories	4 ?
Distinct scripts	2 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	Natural gas
2nd row	Natural gas
3rd row	Natural gas
4th row	Hydro
5th row	Natural gas

Common Values

Value	Count	Frequency (%)
Natural gas	8356	16.7%
Hydro	7560	15.1%
Solar	7356	14.7%
Other	6452	12.9%
Coal	6318	12.6%
Wind	5665	11.3%
Petroleum	4159	8.3%
Nuclear	4134	8.3%
type-name	9	< 0.1%

Length

Histogram of lengths of the category

Category Frequency Plot

Value	Count	Frequency (%)
natural	8356	14.3%
gas	8356	14.3%
hydro	7560	13.0%
solar	7356	12.6%
other	6452	11.1%
coal	6318	10.8%
wind	5665	9.7%
petroleum	4159	7.1%
nuclear	4134	7.1%
type-name	9	< 0.1%

Most occurring characters

Value	Count	Frequency (%)
a	42885	13.7%
r	38017	12.1%
l	30323	9.7%
o	25393	8.1%
t	18976	6.1%
e	18922	6.0%
u	16649	5.3%
d	13225	4.2%
N	12490	4.0%
	8356	2.7%
Other values (16)	87902	28.1%

Most occurring categories

Value	Count	Frequency (%)
Lowercase Letter	254773	81.4%
Uppercase Letter	50000	16.0%
Space Separator	8356	2.7%
Dash Punctuation	9	< 0.1%

Most frequent character per category

Lowercase Letter

Value	Count	Frequency (%)
a	42885	16.8%
r	38017	14.9%
l	30323	11.9%
o	25393	10.0%
t	18976	7.4%
e	18922	7.4%
u	16649	6.5%
d	13225	5.2%
g	8356	3.3%
s	8356	3.3%
Other values (7)	33671	13.2%

Uppercase Letter

Value	Count	Frequency (%)
N	12490	25.0%
H	7560	15.1%
S	7356	14.7%
O	6452	12.9%
C	6318	12.6%
W	5665	11.3%
P	4159	8.3%

Space Separator

Value	Count	Frequency (%)
	8356	100.0%

Dash Punctuation

Value	Count	Frequency (%)
-	9	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Latin	304773	97.3%
Common	8365	2.7%

Most frequent character per script

Latin

Value	Count	Frequency (%)
a	42885	14.1%
r	38017	12.5%
l	30323	9.9%
o	25393	8.3%
t	18976	6.2%
e	18922	6.2%
u	16649	5.5%
d	13225	4.3%
N	12490	4.1%
g	8356	2.7%
Other values (14)	79537	26.1%

Common

Value	Count	Frequency (%)
	8356	99.9%
-	9	0.1%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	313138	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
a	42885	13.7%
r	38017	12.1%
l	30323	9.7%
o	25393	8.1%
t	18976	6.1%
e	18922	6.0%
u	16649	5.3%
d	13225	4.2%
N	12490	4.0%
	8356	2.7%
Other values (16)	87902	28.1%

timezone
Categorical

HIGH CORRELATION

Distinct	6
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	390.8 KiB

Pacific	10064
Mountain	10020
Arizona	10015
Eastern	9953
Central	9948

Length

Max length	8
Median length	7
Mean length	7.200543902
Min length	7

Characters and Unicode

Total characters	360092
Distinct characters	19
Distinct categories	2 ?
Distinct scripts	1 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	Mountain
2nd row	Eastern
3rd row	Mountain
4th row	Pacific
5th row	Central

Common Values

Value	Count	Frequency (%)
Pacific	10064	20.1%
Mountain	10020	20.0%
Arizona	10015	20.0%
Eastern	9953	19.9%
Central	9948	19.9%
timezone	9	< 0.1%

Length

Histogram of lengths of the category

Category Frequency Plot

Value	Count	Frequency (%)
pacific	10064	20.1%
mountain	10020	20.0%
arizona	10015	20.0%
eastern	9953	19.9%
central	9948	19.9%
timezone	9	< 0.1%

Most occurring characters

Value	Count	Frequency (%)
a	50000	13.9%
n	49965	13.9%
i	40172	11.2%
t	29930	8.3%
r	29916	8.3%
c	20128	5.6%
o	20044	5.6%
e	19919	5.5%
P	10064	2.8%
f	10064	2.8%
Other values (9)	79890	22.2%

Most occurring categories

Value	Count	Frequency (%)
Lowercase Letter	310092	86.1%
Uppercase Letter	50000	13.9%

Most frequent character per category

Lowercase Letter

Value	Count	Frequency (%)
a	50000	16.1%
n	49965	16.1%
i	40172	13.0%
t	29930	9.7%
r	29916	9.6%
c	20128	6.5%
o	20044	6.5%
e	19919	6.4%
f	10064	3.2%
z	10024	3.2%
Other values (4)	29930	9.7%

Uppercase Letter

Value	Count	Frequency (%)
P	10064	20.1%
M	10020	20.0%
A	10015	20.0%
E	9953	19.9%
C	9948	19.9%

Most occurring scripts

Value	Count	Frequency (%)
Latin	360092	100.0%

Most frequent character per script

Latin

Value	Count	Frequency (%)
a	50000	13.9%
n	49965	13.9%
i	40172	11.2%
t	29930	8.3%
r	29916	8.3%
c	20128	5.6%
o	20044	5.6%
e	19919	5.5%
P	10064	2.8%
f	10064	2.8%
Other values (9)	79890	22.2%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	360092	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
a	50000	13.9%
n	49965	13.9%
i	40172	11.2%
t	29930	8.3%
r	29916	8.3%
c	20128	5.6%
o	20044	5.6%
e	19919	5.5%
P	10064	2.8%
f	10064	2.8%
Other values (9)	79890	22.2%

timezone-description
Categorical

HIGH CORRELATION

Distinct	6
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	390.8 KiB

Pacific	10064
Mountain	10020
Arizona	10015
Eastern	9953
Central	9948

Length

Max length	20
Median length	7
Mean length	7.202703513
Min length	7

Characters and Unicode

Total characters	360200
Distinct characters	22
Distinct categories	3 ?
Distinct scripts	2 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	Mountain
2nd row	Eastern
3rd row	Mountain
4th row	Pacific
5th row	Central

Common Values

Value	Count	Frequency (%)
Pacific	10064	20.1%
Mountain	10020	20.0%
Arizona	10015	20.0%
Eastern	9953	19.9%
Central	9948	19.9%
timezone-description	9	< 0.1%

Length

Histogram of lengths of the category

Category Frequency Plot

Value	Count	Frequency (%)
pacific	10064	20.1%
mountain	10020	20.0%
arizona	10015	20.0%
eastern	9953	19.9%
central	9948	19.9%
timezone-description	9	< 0.1%

Most occurring characters

Value	Count	Frequency (%)
a	50000	13.9%
n	49974	13.9%
i	40190	11.2%
t	29939	8.3%
r	29925	8.3%
c	20137	5.6%
o	20053	5.6%
e	19928	5.5%
P	10064	2.8%
f	10064	2.8%
Other values (12)	79926	22.2%

Most occurring categories

Value	Count	Frequency (%)
Lowercase Letter	310191	86.1%
Uppercase Letter	50000	13.9%
Dash Punctuation	9	< 0.1%

Most frequent character per category

Lowercase Letter

Value	Count	Frequency (%)
a	50000	16.1%
n	49974	16.1%
i	40190	13.0%
t	29939	9.7%
r	29925	9.6%
c	20137	6.5%
o	20053	6.5%
e	19928	6.4%
f	10064	3.2%
z	10024	3.2%
Other values (6)	29957	9.7%

Uppercase Letter

Value	Count	Frequency (%)
P	10064	20.1%
M	10020	20.0%
A	10015	20.0%
E	9953	19.9%
C	9948	19.9%

Dash Punctuation

Value	Count	Frequency (%)
-	9	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Latin	360191	> 99.9%
Common	9	< 0.1%

Most frequent character per script

Latin

Value	Count	Frequency (%)
a	50000	13.9%
n	49974	13.9%
i	40190	11.2%
t	29939	8.3%
r	29925	8.3%
c	20137	5.6%
o	20053	5.6%
e	19928	5.5%
P	10064	2.8%
f	10064	2.8%
Other values (11)	79917	22.2%

Common

Value	Count	Frequency (%)
-	9	100.0%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	360200	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
a	50000	13.9%
n	49974	13.9%
i	40190	11.2%
t	29939	8.3%
r	29925	8.3%
c	20137	5.6%
o	20053	5.6%
e	19928	5.5%
P	10064	2.8%
f	10064	2.8%
Other values (12)	79926	22.2%

value
Categorical

HIGH CARDINALITY

Distinct	19112
Distinct (%)	38.2%
Missing	0
Missing (%)	0.0%
Memory size	390.8 KiB

0	4971
-3	231
24	112
-48	108
48	94
Other values (19107)	44493

Length

Max length	7
Median length	6
Mean length	4.114399408
Min length	1

Characters and Unicode

Total characters	205757
Distinct characters	16
Distinct categories	3 ?
Distinct scripts	2 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	8656 ?
Unique (%)	17.3%

Sample

1st row	35238
2nd row	35360
3rd row	2508
4th row	11272
5th row	2476

Common Values

Value	Count	Frequency (%)
0	4971	9.9%
-3	231	0.5%
24	112	0.2%
-48	108	0.2%
48	94	0.2%
-2	84	0.2%
-11	76	0.2%
1	74	0.1%
4	71	0.1%
-10	65	0.1%
Other values (19102)	44123	88.2%

Length

Histogram of lengths of the category

Value	Count	Frequency (%)
0	4971	9.9%
3	272	0.5%
48	202	0.4%
2	132	0.3%
1	121	0.2%
24	112	0.2%
4	104	0.2%
10	104	0.2%
14	104	0.2%
11	103	0.2%
Other values (18928)	43784	87.6%

Most occurring characters

Value	Count	Frequency (%)
1	30006	14.6%
2	24380	11.8%
0	21525	10.5%
3	20895	10.2%
4	19862	9.7%
5	18399	8.9%
6	18174	8.8%
7	18002	8.7%
8	16588	8.1%
9	16487	8.0%
Other values (6)	1439	0.7%

Most occurring categories

Value	Count	Frequency (%)
Decimal Number	204318	99.3%
Dash Punctuation	1394	0.7%
Lowercase Letter	45	< 0.1%

Most frequent character per category

Decimal Number

Value	Count	Frequency (%)
1	30006	14.7%
2	24380	11.9%
0	21525	10.5%
3	20895	10.2%
4	19862	9.7%
5	18399	9.0%
6	18174	8.9%
7	18002	8.8%
8	16588	8.1%
9	16487	8.1%

Lowercase Letter

Value	Count	Frequency (%)
v	9	20.0%
a	9	20.0%
l	9	20.0%
u	9	20.0%
e	9	20.0%

Dash Punctuation

Value	Count	Frequency (%)
-	1394	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Common	205712	> 99.9%
Latin	45	< 0.1%

Most frequent character per script

Common

Value	Count	Frequency (%)
1	30006	14.6%
2	24380	11.9%
0	21525	10.5%
3	20895	10.2%
4	19862	9.7%
5	18399	8.9%
6	18174	8.8%
7	18002	8.8%
8	16588	8.1%
9	16487	8.0%

Latin

Value	Count	Frequency (%)
v	9	20.0%
a	9	20.0%
l	9	20.0%
u	9	20.0%
e	9	20.0%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	205757	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
1	30006	14.6%
2	24380	11.8%
0	21525	10.5%
3	20895	10.2%
4	19862	9.7%
5	18399	8.9%
6	18174	8.8%
7	18002	8.7%
8	16588	8.1%
9	16487	8.0%
Other values (6)	1439	0.7%

value-units
Categorical

HIGH CORRELATION

Distinct	2
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	390.8 KiB

megawatthours	50000
value-units	9

Length

Max length	13
Median length	13
Mean length	12.99964006
Min length	11

Characters and Unicode

Total characters	650099
Distinct characters	16
Distinct categories	2 ?
Distinct scripts	2 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	megawatthours
2nd row	megawatthours
3rd row	megawatthours
4th row	megawatthours
5th row	megawatthours

Common Values

Value	Count	Frequency (%)
megawatthours	50000	> 99.9%
value-units	9	< 0.1%

Length

Histogram of lengths of the category

Category Frequency Plot

Value	Count	Frequency (%)
megawatthours	50000	> 99.9%
value-units	9	< 0.1%

Most occurring characters

Value	Count	Frequency (%)
a	100009	15.4%
t	100009	15.4%
u	50018	7.7%
e	50009	7.7%
s	50009	7.7%
m	50000	7.7%
g	50000	7.7%
w	50000	7.7%
h	50000	7.7%
o	50000	7.7%
Other values (6)	50045	7.7%

Most occurring categories

Value	Count	Frequency (%)
Lowercase Letter	650090	> 99.9%
Dash Punctuation	9	< 0.1%

Most frequent character per category

Lowercase Letter

Value	Count	Frequency (%)
a	100009	15.4%
t	100009	15.4%
u	50018	7.7%
e	50009	7.7%
s	50009	7.7%
m	50000	7.7%
g	50000	7.7%
w	50000	7.7%
h	50000	7.7%
o	50000	7.7%
Other values (5)	50036	7.7%

Dash Punctuation

Value	Count	Frequency (%)
-	9	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Latin	650090	> 99.9%
Common	9	< 0.1%

Most frequent character per script

Latin

Value	Count	Frequency (%)
a	100009	15.4%
t	100009	15.4%
u	50018	7.7%
e	50009	7.7%
s	50009	7.7%
m	50000	7.7%
g	50000	7.7%
w	50000	7.7%
h	50000	7.7%
o	50000	7.7%
Other values (5)	50036	7.7%

Common

Value	Count	Frequency (%)
-	9	100.0%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	650099	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
a	100009	15.4%
t	100009	15.4%
u	50018	7.7%
e	50009	7.7%
s	50009	7.7%
m	50000	7.7%
g	50000	7.7%
w	50000	7.7%
h	50000	7.7%
o	50000	7.7%
Other values (6)	50045	7.7%

Unnamed: 0

Unnamed: 0

Auto

The auto setting is an easily interpretable pairwise column metric of the following mapping: vartype-vartype : method, categorical-categorical : Cramer's V, numerical-categorical : Cramer's V (using a discretized numerical column), numerical-numerical : Spearman's ρ. This configuration uses the best suitable for each pair of columns.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Cramér's V (φc)

Cramér's V is an association measure for nominal random variables. The coefficient ranges from 0 to 1, with 0 indicating independence and 1 indicating perfect association. The empirical estimators used for Cramér's V have been proved to be biased, even for large samples. We use a bias-corrected measure that has been proposed by Bergsma in 2013 that can be found here.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

The dendrogram allows you to more fully correlate variable completion, revealing trends deeper than the pairwise ones visible in the correlation heatmap.

First rows

	Unnamed: 0	period	respondent	respondent-name	fueltype	type-name	timezone	timezone-description	value	value-units
0	0.0	2022-11-13	PACE	PacifiCorp East	NG	Natural gas	Mountain	Mountain	35238	megawatthours
1	1.0	2022-11-13	PACE	PacifiCorp East	NG	Natural gas	Eastern	Eastern	35360	megawatthours
2	2.0	2022-11-13	IID	Imperial Irrigation District	NG	Natural gas	Mountain	Mountain	2508	megawatthours
3	3.0	2022-11-13	IPCO	Idaho Power Company	WAT	Hydro	Pacific	Pacific	11272	megawatthours
4	4.0	2022-11-13	IID	Imperial Irrigation District	NG	Natural gas	Central	Central	2476	megawatthours
5	5.0	2022-11-13	CPLW	Duke Energy Progress West	COL	Coal	Eastern	Eastern	0	megawatthours
6	6.0	2022-11-13	SCL	Seattle City Light	WAT	Hydro	Arizona	Arizona	12100	megawatthours
7	7.0	2022-11-13	BPAT	Bonneville Power Administration	WND	Wind	Pacific	Pacific	1263	megawatthours
8	8.0	2022-11-13	CISO	California Independent System Operator	WAT	Hydro	Mountain	Mountain	19652	megawatthours
9	9.0	2022-11-13	SC	South Carolina Public Service Authority	OIL	Petroleum	Eastern	Eastern	-2	megawatthours

Last rows

	Unnamed: 0	period	respondent	respondent-name	fueltype	type-name	timezone	timezone-description	value	value-units
49999	4990.0	2022-10-18	JEA	JEA	SUN	Solar	Eastern	Eastern	247	megawatthours
50000	4991.0	2022-10-18	AECI	Associated Electric Cooperative, Inc.	COL	Coal	Central	Central	33982	megawatthours
50001	4992.0	2022-10-18	ISNE	ISO New England	OIL	Petroleum	Eastern	Eastern	573	megawatthours
50002	4993.0	2022-10-18	FPL	Florida Power & Light Co.	SUN	Solar	Eastern	Eastern	18041	megawatthours
50003	4994.0	2022-10-18	AZPS	Arizona Public Service Company	WND	Wind	Arizona	Arizona	330	megawatthours
50004	4995.0	2022-10-18	TEN	Tennessee	OTH	Other	Central	Central	162	megawatthours
50005	4996.0	2022-10-18	AZPS	Arizona Public Service Company	WAT	Hydro	Central	Central	0	megawatthours
50006	4997.0	2022-10-18	NEVP	Nevada Power Company	SUN	Solar	Mountain	Mountain	13370	megawatthours
50007	4998.0	2022-10-18	PACW	PacifiCorp West	WND	Wind	Eastern	Eastern	-12	megawatthours
50008	4999.0	2022-10-18	PSCO	Public Service Company of Colorado	NG	Natural gas	Mountain	Mountain	43363	megawatthours

Most frequently occurring

	Unnamed: 0	period	respondent	respondent-name	fueltype	type-name	timezone	timezone-description	value	value-units	# duplicates
0	NaN	period	respondent	respondent-name	fueltype	type-name	timezone	timezone-description	value	value-units	9

Overview

Variables

Common Values

Length

Most occurring characters

Most occurring categories

Most frequent character per category

Decimal Number

Lowercase Letter

Dash Punctuation

Most occurring scripts

Most frequent character per script

Common

Latin

Most occurring blocks

Most frequent character per block

ASCII

Common Values

Length

Most occurring characters

Most occurring categories

Most frequent character per category

Uppercase Letter

Lowercase Letter

Decimal Number

Most occurring scripts

Most frequent character per script

Latin

Common

Most occurring blocks

Most frequent character per block

ASCII

Common Values

Length

Most occurring characters

Most occurring categories

Most frequent character per category

Lowercase Letter

Uppercase Letter

Decimal Number

Other Punctuation

Space Separator

Dash Punctuation

Most occurring scripts

Most frequent character per script

Latin

Common

Most occurring blocks

Most frequent character per block

ASCII

Common Values

Length

Category Frequency Plot

Most occurring characters

Most occurring categories

Most frequent character per category

Uppercase Letter

Lowercase Letter

Most occurring scripts

Most frequent character per script

Latin

Most occurring blocks

Most frequent character per block

ASCII

Common Values

Length

Category Frequency Plot

Most occurring characters

Most occurring categories

Most frequent character per category

Lowercase Letter

Uppercase Letter

Space Separator

Dash Punctuation

Most occurring scripts

Most frequent character per script

Latin

Common

Most occurring blocks

Most frequent character per block