Pandas Profiling Report

Dataset statistics

Number of variables	9
Number of observations	50009
Missing cells	9
Missing cells (%)	< 0.1%
Duplicate rows	1
Duplicate rows (%)	< 0.1%
Total size in memory	3.4 MiB
Average record size in memory	72.0 B

Variable types

Numeric	1
Categorical	8

Alerts

Dataset has 1 (< 0.1%) duplicate rows	Duplicates
`fromba` has a high cardinality: 77 distinct values	High cardinality
`fromba-name` has a high cardinality: 77 distinct values	High cardinality
`toba` has a high cardinality: 86 distinct values	High cardinality
`toba-name` has a high cardinality: 86 distinct values	High cardinality
`value` has a high cardinality: 21249 distinct values	High cardinality
`Unnamed: 0` is highly correlated with `period`	High correlation
`period` is highly correlated with `Unnamed: 0` and 6 other fields	High correlation
`fromba` is highly correlated with `period` and 5 other fields	High correlation
`fromba-name` is highly correlated with `period` and 5 other fields	High correlation
`toba` is highly correlated with `period` and 5 other fields	High correlation
`toba-name` is highly correlated with `period` and 5 other fields	High correlation
`timezone` is highly correlated with `period` and 5 other fields	High correlation
`value-units` is highly correlated with `period` and 5 other fields	High correlation
`Unnamed: 0` is uniformly distributed	Uniform

Reproduction

Analysis started	2022-11-17 22:36:01.257580
Analysis finished	2022-11-17 22:36:06.082039
Duration	4.82 seconds
Software version	pandas-profiling v3.4.0
Download configuration	config.json

Unnamed: 0
Real number (ℝ_≥0)

HIGH CORRELATION
UNIFORM

Distinct	5000
Distinct (%)	10.0%
Missing	9
Missing (%)	< 0.1%
Infinite	0
Infinite (%)	0.0%
Mean	2499.5

Minimum	0
Maximum	4999
Zeros	10
Zeros (%)	< 0.1%
Negative	0
Negative (%)	0.0%
Memory size	390.8 KiB

Quantile statistics

Minimum	0
5-th percentile	249.95
Q1	1249.75
median	2499.5
Q3	3749.25
95-th percentile	4749.05
Maximum	4999
Range	4999
Interquartile range (IQR)	2499.5

Descriptive statistics

Standard deviation	1443.390078
Coefficient of variation (CV)	0.5774715255
Kurtosis	-1.200000095
Mean	2499.5
Median Absolute Deviation (MAD)	1250
Skewness	0
Sum	124975000
Variance	2083374.917
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	10	< 0.1%
3331	10	< 0.1%
3338	10	< 0.1%
3337	10	< 0.1%
3336	10	< 0.1%
3335	10	< 0.1%
3334	10	< 0.1%
3333	10	< 0.1%
3332	10	< 0.1%
3330	10	< 0.1%
Other values (4990)	49900	99.8%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
0	10	< 0.1%
1	10	< 0.1%
2	10	< 0.1%
3	10	< 0.1%
4	10	< 0.1%
5	10	< 0.1%
6	10	< 0.1%
7	10	< 0.1%
8	10	< 0.1%
9	10	< 0.1%

Value	Count	Frequency (%)
4999	10	< 0.1%
4998	10	< 0.1%
4997	10	< 0.1%
4996	10	< 0.1%
4995	10	< 0.1%
4994	10	< 0.1%
4993	10	< 0.1%
4992	10	< 0.1%
4991	10	< 0.1%
4990	10	< 0.1%

period
Categorical

HIGH CORRELATION

Distinct	32
Distinct (%)	0.1%
Missing	0
Missing (%)	0.0%
Memory size	390.8 KiB

2022-10-28	1700
2022-10-15	1700
2022-10-17	1700
2022-10-18	1700
2022-10-19	1700
Other values (27)	41509

Length

Max length	10
Median length	10
Mean length	9.99928013
Min length	6

Characters and Unicode

Total characters	500054
Distinct characters	17
Distinct categories	3 ?
Distinct scripts	2 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	2022-11-12
2nd row	2022-11-12
3rd row	2022-11-12
4th row	2022-11-12
5th row	2022-11-12

Common Values

Value	Count	Frequency (%)
2022-10-28	1700	3.4%
2022-10-15	1700	3.4%
2022-10-17	1700	3.4%
2022-10-18	1700	3.4%
2022-10-19	1700	3.4%
2022-10-20	1700	3.4%
2022-10-21	1700	3.4%
2022-10-22	1700	3.4%
2022-10-23	1700	3.4%
2022-10-24	1700	3.4%
Other values (22)	33009	66.0%

Length

Histogram of lengths of the category

Value	Count	Frequency (%)
2022-10-28	1700	3.4%
2022-10-26	1700	3.4%
2022-11-09	1700	3.4%
2022-10-14	1700	3.4%
2022-11-03	1700	3.4%
2022-11-02	1700	3.4%
2022-10-15	1700	3.4%
2022-10-31	1700	3.4%
2022-10-30	1700	3.4%
2022-10-29	1700	3.4%
Other values (22)	33009	66.0%

Most occurring characters

Value	Count	Frequency (%)
2	171353	34.3%
0	100862	20.2%
-	100000	20.0%
1	90468	18.1%
3	7021	1.4%
9	5100	1.0%
8	5081	1.0%
4	5041	1.0%
6	5032	1.0%
7	5029	1.0%
Other values (7)	5067	1.0%

Most occurring categories

Value	Count	Frequency (%)
Decimal Number	400000	80.0%
Dash Punctuation	100000	20.0%
Lowercase Letter	54	< 0.1%

Most frequent character per category

Decimal Number

Value	Count	Frequency (%)
2	171353	42.8%
0	100862	25.2%
1	90468	22.6%
3	7021	1.8%
9	5100	1.3%
8	5081	1.3%
4	5041	1.3%
6	5032	1.3%
7	5029	1.3%
5	5013	1.3%

Lowercase Letter

Value	Count	Frequency (%)
p	9	16.7%
e	9	16.7%
r	9	16.7%
i	9	16.7%
o	9	16.7%
d	9	16.7%

Dash Punctuation

Value	Count	Frequency (%)
-	100000	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Common	500000	> 99.9%
Latin	54	< 0.1%

Most frequent character per script

Common

Value	Count	Frequency (%)
2	171353	34.3%
0	100862	20.2%
-	100000	20.0%
1	90468	18.1%
3	7021	1.4%
9	5100	1.0%
8	5081	1.0%
4	5041	1.0%
6	5032	1.0%
7	5029	1.0%

Latin

Value	Count	Frequency (%)
p	9	16.7%
e	9	16.7%
r	9	16.7%
i	9	16.7%
o	9	16.7%
d	9	16.7%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	500054	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
2	171353	34.3%
0	100862	20.2%
-	100000	20.0%
1	90468	18.1%
3	7021	1.4%
9	5100	1.0%
8	5081	1.0%
4	5041	1.0%
6	5032	1.0%
7	5029	1.0%
Other values (7)	5067	1.0%

fromba
Categorical

HIGH CARDINALITY
HIGH CORRELATION

Distinct	77
Distinct (%)	0.2%
Missing	0
Missing (%)	0.0%
Memory size	390.8 KiB

BPAT	2856
CISO	1737
AZPS	1390
SWPP	1386
WALC	1362
Other values (72)	41278

Length

Max length	6
Median length	4
Mean length	3.577436062
Min length	2

Characters and Unicode

Total characters	178904
Distinct characters	33
Distinct categories	3 ?
Distinct scripts	2 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	PACE
2nd row	SPA
3rd row	BPAT
4th row	CISO
5th row	TEN

Common Values

Value	Count	Frequency (%)
BPAT	2856	5.7%
CISO	1737	3.5%
AZPS	1390	2.8%
SWPP	1386	2.8%
WALC	1362	2.7%
SOCO	1274	2.5%
DUK	1245	2.5%
NWMT	1240	2.5%
FPL	1107	2.2%
PACW	1098	2.2%
Other values (67)	35314	70.6%

Length

Histogram of lengths of the category

Value	Count	Frequency (%)
bpat	2856	5.7%
ciso	1737	3.5%
azps	1390	2.8%
swpp	1386	2.8%
walc	1362	2.7%
soco	1274	2.5%
duk	1245	2.5%
nwmt	1240	2.5%
fpl	1107	2.2%
pacw	1098	2.2%
Other values (67)	35314	70.6%

Most occurring characters

Value	Count	Frequency (%)
P	24872	13.9%
C	19050	10.6%
A	17638	9.9%
S	14879	8.3%
E	12918	7.2%
W	10857	6.1%
I	9144	5.1%
T	9046	5.1%
N	7575	4.2%
O	7295	4.1%
Other values (23)	45630	25.5%

Most occurring categories

Value	Count	Frequency (%)
Uppercase Letter	178286	99.7%
Decimal Number	564	0.3%
Lowercase Letter	54	< 0.1%

Most frequent character per category

Uppercase Letter

Value	Count	Frequency (%)
P	24872	14.0%
C	19050	10.7%
A	17638	9.9%
S	14879	8.3%
E	12918	7.2%
W	10857	6.1%
I	9144	5.1%
T	9046	5.1%
N	7575	4.2%
O	7295	4.1%
Other values (15)	45012	25.2%

Lowercase Letter

Value	Count	Frequency (%)
f	9	16.7%
r	9	16.7%
o	9	16.7%
m	9	16.7%
b	9	16.7%
a	9	16.7%

Decimal Number

Value	Count	Frequency (%)
4	282	50.0%
8	282	50.0%

Most occurring scripts

Value	Count	Frequency (%)
Latin	178340	99.7%
Common	564	0.3%

Most frequent character per script

Latin

Value	Count	Frequency (%)
P	24872	13.9%
C	19050	10.7%
A	17638	9.9%
S	14879	8.3%
E	12918	7.2%
W	10857	6.1%
I	9144	5.1%
T	9046	5.1%
N	7575	4.2%
O	7295	4.1%
Other values (21)	45066	25.3%

Common

Value	Count	Frequency (%)
4	282	50.0%
8	282	50.0%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	178904	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
P	24872	13.9%
C	19050	10.6%
A	17638	9.9%
S	14879	8.3%
E	12918	7.2%
W	10857	6.1%
I	9144	5.1%
T	9046	5.1%
N	7575	4.2%
O	7295	4.1%
Other values (23)	45630	25.5%

fromba-name
Categorical

HIGH CARDINALITY
HIGH CORRELATION

Distinct	77
Distinct (%)	0.2%
Missing	0
Missing (%)	0.0%
Memory size	390.8 KiB

Bonneville Power Administration	2856
California Independent System Operator	1737
Arizona Public Service Company	1390
Southwest Power Pool	1386
Western Area Power Administration - Desert Southwest Region	1362
Other values (72)	41278

Length

Max length	66
Median length	38
Mean length	29.22353976
Min length	3

Characters and Unicode

Total characters	1461440
Distinct characters	58
Distinct categories	6 ?
Distinct scripts	2 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	PacifiCorp East
2nd row	Southwestern Power Administration
3rd row	Bonneville Power Administration
4th row	California Independent System Operator
5th row	Tennessee

Common Values

Value	Count	Frequency (%)
Bonneville Power Administration	2856	5.7%
California Independent System Operator	1737	3.5%
Arizona Public Service Company	1390	2.8%
Southwest Power Pool	1386	2.8%
Western Area Power Administration - Desert Southwest Region	1362	2.7%
Southern Company Services, Inc. - Trans	1274	2.5%
Duke Energy Carolinas	1245	2.5%
NorthWestern Corporation	1240	2.5%
Florida Power & Light Co.	1107	2.2%
PacifiCorp West	1098	2.2%
Other values (67)	35314	70.6%

Length

Histogram of lengths of the category

Value	Count	Frequency (%)
power	14899	7.5%
company	8437	4.3%
administration	6752	3.4%
inc	5924	3.0%
of	5737	2.9%
	5544	2.8%
energy	5446	2.8%
public	5169	2.6%
electric	3995	2.0%
service	3611	1.8%
Other values (129)	132369	66.9%

Most occurring characters

Value	Count	Frequency (%)
	147874	10.1%
e	130337	8.9%
o	108872	7.4%
n	106320	7.3%
r	100381	6.9%
i	100358	6.9%
t	98916	6.8%
a	78317	5.4%
s	49065	3.4%
l	45560	3.1%
Other values (48)	495440	33.9%

Most occurring categories

Value	Count	Frequency (%)
Lowercase Letter	1092175	74.7%
Uppercase Letter	195003	13.3%
Space Separator	147874	10.1%
Other Punctuation	19303	1.3%
Dash Punctuation	4977	0.3%
Decimal Number	2108	0.1%

Most frequent character per category

Lowercase Letter

Value	Count	Frequency (%)
e	130337	11.9%
o	108872	10.0%
n	106320	9.7%
r	100381	9.2%
i	100358	9.2%
t	98916	9.1%
a	78317	7.2%
s	49065	4.5%
l	45560	4.2%
c	41611	3.8%
Other values (16)	232438	21.3%

Uppercase Letter

Value	Count	Frequency (%)
P	29427	15.1%
C	27711	14.2%
S	19150	9.8%
A	18380	9.4%
I	13902	7.1%
E	12920	6.6%
D	10888	5.6%
L	8316	4.3%
W	8067	4.1%
N	7979	4.1%
Other values (13)	38263	19.6%

Decimal Number

Value	Count	Frequency (%)
1	918	43.5%
2	626	29.7%
4	282	13.4%
8	282	13.4%

Other Punctuation

Value	Count	Frequency (%)
,	9621	49.8%
.	8575	44.4%
&	1107	5.7%

Space Separator

Value	Count	Frequency (%)
	147874	100.0%

Dash Punctuation

Value	Count	Frequency (%)
-	4977	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Latin	1287178	88.1%
Common	174262	11.9%

Most frequent character per script

Latin

Value	Count	Frequency (%)
e	130337	10.1%
o	108872	8.5%
n	106320	8.3%
r	100381	7.8%
i	100358	7.8%
t	98916	7.7%
a	78317	6.1%
s	49065	3.8%
l	45560	3.5%
c	41611	3.2%
Other values (39)	427441	33.2%

Common

Value	Count	Frequency (%)
	147874	84.9%
,	9621	5.5%
.	8575	4.9%
-	4977	2.9%
&	1107	0.6%
1	918	0.5%
2	626	0.4%
4	282	0.2%
8	282	0.2%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	1461440	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
	147874	10.1%
e	130337	8.9%
o	108872	7.4%
n	106320	7.3%
r	100381	6.9%
i	100358	6.9%
t	98916	6.8%
a	78317	5.4%
s	49065	3.4%
l	45560	3.1%
Other values (48)	495440	33.9%

toba
Categorical

HIGH CARDINALITY
HIGH CORRELATION

Distinct	86
Distinct (%)	0.2%
Missing	0
Missing (%)	0.0%
Memory size	390.8 KiB

BPAT	2653
CISO	1556
AZPS	1397
WALC	1393
SWPP	1303
Other values (81)	41707

Length

Max length	4
Median length	4
Mean length	3.565078286
Min length	2

Characters and Unicode

Total characters	178286
Distinct characters	30
Distinct categories	2 ?
Distinct scripts	1 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	LDWP
2nd row	MISO
3rd row	AVRN
4th row	LDWP
5th row	CAR

Common Values

Value	Count	Frequency (%)
BPAT	2653	5.3%
CISO	1556	3.1%
AZPS	1397	2.8%
WALC	1393	2.8%
SWPP	1303	2.6%
DUK	1247	2.5%
SOCO	1243	2.5%
FPL	1116	2.2%
PACW	1104	2.2%
SRP	1077	2.2%
Other values (76)	35920	71.8%

Length

Histogram of lengths of the category

Value	Count	Frequency (%)
bpat	2653	5.3%
ciso	1556	3.1%
azps	1397	2.8%
walc	1393	2.8%
swpp	1303	2.6%
duk	1247	2.5%
soco	1243	2.5%
fpl	1116	2.2%
pacw	1104	2.2%
srp	1077	2.2%
Other values (76)	35920	71.8%

Most occurring characters

Value	Count	Frequency (%)
P	24658	13.8%
C	19790	11.1%
A	18423	10.3%
S	14227	8.0%
E	13126	7.4%
W	10434	5.9%
T	8613	4.8%
I	8407	4.7%
N	7557	4.2%
M	7517	4.2%
Other values (20)	45534	25.5%

Most occurring categories

Value	Count	Frequency (%)
Uppercase Letter	178250	> 99.9%
Lowercase Letter	36	< 0.1%

Most frequent character per category

Uppercase Letter

Value	Count	Frequency (%)
P	24658	13.8%
C	19790	11.1%
A	18423	10.3%
S	14227	8.0%
E	13126	7.4%
W	10434	5.9%
T	8613	4.8%
I	8407	4.7%
N	7557	4.2%
M	7517	4.2%
Other values (16)	45498	25.5%

Lowercase Letter

Value	Count	Frequency (%)
t	9	25.0%
o	9	25.0%
b	9	25.0%
a	9	25.0%

Most occurring scripts

Value	Count	Frequency (%)
Latin	178286	100.0%

Most frequent character per script

Latin

Value	Count	Frequency (%)
P	24658	13.8%
C	19790	11.1%
A	18423	10.3%
S	14227	8.0%
E	13126	7.4%
W	10434	5.9%
T	8613	4.8%
I	8407	4.7%
N	7557	4.2%
M	7517	4.2%
Other values (20)	45534	25.5%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	178286	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
P	24658	13.8%
C	19790	11.1%
A	18423	10.3%
S	14227	8.0%
E	13126	7.4%
W	10434	5.9%
T	8613	4.8%
I	8407	4.7%
N	7557	4.2%
M	7517	4.2%
Other values (20)	45534	25.5%

toba-name
Categorical

HIGH CARDINALITY
HIGH CORRELATION

Distinct	86
Distinct (%)	0.2%
Missing	0
Missing (%)	0.0%
Memory size	390.8 KiB

Bonneville Power Administration	2653
California Independent System Operator	1556
Arizona Public Service Company	1397
Western Area Power Administration - Desert Southwest Region	1393
Southwest Power Pool	1303
Other values (81)	41707

Length

Max length	66
Median length	37
Mean length	28.98570257
Min length	3

Characters and Unicode

Total characters	1449546
Distinct characters	57
Distinct categories	6 ?
Distinct scripts	2 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	Los Angeles Department of Water and Power
2nd row	Midcontinent Independent System Operator, Inc.
3rd row	Avangrid Renewables, LLC
4th row	Los Angeles Department of Water and Power
5th row	Carolinas

Common Values

Value	Count	Frequency (%)
Bonneville Power Administration	2653	5.3%
California Independent System Operator	1556	3.1%
Arizona Public Service Company	1397	2.8%
Western Area Power Administration - Desert Southwest Region	1393	2.8%
Southwest Power Pool	1303	2.6%
Duke Energy Carolinas	1247	2.5%
Southern Company Services, Inc. - Trans	1243	2.5%
Florida Power & Light Co.	1116	2.2%
PacifiCorp West	1104	2.2%
Salt River Project Agricultural Improvement and Power District	1077	2.2%
Other values (76)	35920	71.8%

Length

Histogram of lengths of the category

Value	Count	Frequency (%)
power	14820	7.5%
company	8305	4.2%
administration	6472	3.3%
of	5704	2.9%
inc	5639	2.9%
	5506	2.8%
energy	5428	2.8%
public	5181	2.6%
electric	3886	2.0%
service	3613	1.8%
Other values (142)	131964	67.2%

Most occurring characters

Value	Count	Frequency (%)
	146509	10.1%
e	127352	8.8%
o	108828	7.5%
n	104556	7.2%
r	100542	6.9%
i	99848	6.9%
t	96586	6.7%
a	80742	5.6%
s	47864	3.3%
l	44574	3.1%
Other values (47)	492145	34.0%

Most occurring categories

Value	Count	Frequency (%)
Lowercase Letter	1083197	74.7%
Uppercase Letter	194159	13.4%
Space Separator	146509	10.1%
Other Punctuation	18861	1.3%
Dash Punctuation	5251	0.4%
Decimal Number	1569	0.1%

Most frequent character per category

Lowercase Letter

Value	Count	Frequency (%)
e	127352	11.8%
o	108828	10.0%
n	104556	9.7%
r	100542	9.3%
i	99848	9.2%
t	96586	8.9%
a	80742	7.5%
s	47864	4.4%
l	44574	4.1%
c	42165	3.9%
Other values (16)	230140	21.2%

Uppercase Letter

Value	Count	Frequency (%)
P	29367	15.1%
C	28654	14.8%
S	18484	9.5%
A	18428	9.5%
E	13248	6.8%
I	13024	6.7%
D	11018	5.7%
L	8097	4.2%
W	7974	4.1%
N	7312	3.8%
Other values (14)	38553	19.9%

Other Punctuation

Value	Count	Frequency (%)
,	9421	49.9%
.	8324	44.1%
&	1116	5.9%

Decimal Number

Value	Count	Frequency (%)
1	935	59.6%
2	634	40.4%

Space Separator

Value	Count	Frequency (%)
	146509	100.0%

Dash Punctuation

Value	Count	Frequency (%)
-	5251	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Latin	1277356	88.1%
Common	172190	11.9%

Most frequent character per script

Latin

Value	Count	Frequency (%)
e	127352	10.0%
o	108828	8.5%
n	104556	8.2%
r	100542	7.9%
i	99848	7.8%
t	96586	7.6%
a	80742	6.3%
s	47864	3.7%
l	44574	3.5%
c	42165	3.3%
Other values (40)	424299	33.2%

Common

Value	Count	Frequency (%)
	146509	85.1%
,	9421	5.5%
.	8324	4.8%
-	5251	3.0%
&	1116	0.6%
1	935	0.5%
2	634	0.4%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	1449546	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
	146509	10.1%
e	127352	8.8%
o	108828	7.5%
n	104556	7.2%
r	100542	6.9%
i	99848	6.9%
t	96586	6.7%
a	80742	5.6%
s	47864	3.3%
l	44574	3.1%
Other values (47)	492145	34.0%

timezone
Categorical

HIGH CORRELATION

Distinct	6
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	390.8 KiB

Eastern	10534
Central	10142
Mountain	9893
Arizona	9819
Pacific	9612

Length

Max length	8
Median length	7
Mean length	7.198004359
Min length	7

Characters and Unicode

Total characters	359965
Distinct characters	19
Distinct categories	2 ?
Distinct scripts	1 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	Mountain
2nd row	Eastern
3rd row	Eastern
4th row	Eastern
5th row	Central

Common Values

Value	Count	Frequency (%)
Eastern	10534	21.1%
Central	10142	20.3%
Mountain	9893	19.8%
Arizona	9819	19.6%
Pacific	9612	19.2%
timezone	9	< 0.1%

Length

Histogram of lengths of the category

Category Frequency Plot

Value	Count	Frequency (%)
eastern	10534	21.1%
central	10142	20.3%
mountain	9893	19.8%
arizona	9819	19.6%
pacific	9612	19.2%
timezone	9	< 0.1%

Most occurring characters

Value	Count	Frequency (%)
n	50290	14.0%
a	50000	13.9%
i	38945	10.8%
t	30578	8.5%
r	30495	8.5%
e	20694	5.7%
o	19721	5.5%
c	19224	5.3%
E	10534	2.9%
s	10534	2.9%
Other values (9)	78950	21.9%

Most occurring categories

Value	Count	Frequency (%)
Lowercase Letter	309965	86.1%
Uppercase Letter	50000	13.9%

Most frequent character per category

Lowercase Letter

Value	Count	Frequency (%)
n	50290	16.2%
a	50000	16.1%
i	38945	12.6%
t	30578	9.9%
r	30495	9.8%
e	20694	6.7%
o	19721	6.4%
c	19224	6.2%
s	10534	3.4%
l	10142	3.3%
Other values (4)	29342	9.5%

Uppercase Letter

Value	Count	Frequency (%)
E	10534	21.1%
C	10142	20.3%
M	9893	19.8%
A	9819	19.6%
P	9612	19.2%

Most occurring scripts

Value	Count	Frequency (%)
Latin	359965	100.0%

Most frequent character per script

Latin

Value	Count	Frequency (%)
n	50290	14.0%
a	50000	13.9%
i	38945	10.8%
t	30578	8.5%
r	30495	8.5%
e	20694	5.7%
o	19721	5.5%
c	19224	5.3%
E	10534	2.9%
s	10534	2.9%
Other values (9)	78950	21.9%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	359965	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
n	50290	14.0%
a	50000	13.9%
i	38945	10.8%
t	30578	8.5%
r	30495	8.5%
e	20694	5.7%
o	19721	5.5%
c	19224	5.3%
E	10534	2.9%
s	10534	2.9%
Other values (9)	78950	21.9%

value
Categorical

HIGH CARDINALITY

Distinct	21249
Distinct (%)	42.5%
Missing	0
Missing (%)	0.0%
Memory size	390.8 KiB

0	994
75	28
-75	25
-54	23
1178	21
Other values (21244)	48918

Length

Max length	7
Median length	6
Mean length	4.502089624
Min length	1

Characters and Unicode

Total characters	225145
Distinct characters	16
Distinct categories	3 ?
Distinct scripts	2 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	9270 ?
Unique (%)	18.5%

Sample

1st row	-1224
2nd row	6259
3rd row	-143
4th row	826
5th row	538

Common Values

Value	Count	Frequency (%)
0	994	2.0%
75	28	0.1%
-75	25	< 0.1%
-54	23	< 0.1%
1178	21	< 0.1%
252	21	< 0.1%
-388	21	< 0.1%
-376	21	< 0.1%
73	21	< 0.1%
54	20	< 0.1%
Other values (21239)	48814	97.6%

Length

Histogram of lengths of the category

Value	Count	Frequency (%)
0	994	2.0%
75	53	0.1%
54	43	0.1%
73	40	0.1%
258	39	0.1%
252	37	0.1%
74	36	0.1%
317	35	0.1%
376	32	0.1%
388	29	0.1%
Other values (14244)	48671	97.3%

Most occurring characters

Value	Count	Frequency (%)
1	31335	13.9%
2	25694	11.4%
-	24941	11.1%
3	21555	9.6%
4	19380	8.6%
5	18309	8.1%
6	17448	7.7%
0	17150	7.6%
7	16984	7.5%
8	16613	7.4%
Other values (6)	15736	7.0%

Most occurring categories

Value	Count	Frequency (%)
Decimal Number	200159	88.9%
Dash Punctuation	24941	11.1%
Lowercase Letter	45	< 0.1%

Most frequent character per category

Decimal Number

Value	Count	Frequency (%)
1	31335	15.7%
2	25694	12.8%
3	21555	10.8%
4	19380	9.7%
5	18309	9.1%
6	17448	8.7%
0	17150	8.6%
7	16984	8.5%
8	16613	8.3%
9	15691	7.8%

Lowercase Letter

Value	Count	Frequency (%)
v	9	20.0%
a	9	20.0%
l	9	20.0%
u	9	20.0%
e	9	20.0%

Dash Punctuation

Value	Count	Frequency (%)
-	24941	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Common	225100	> 99.9%
Latin	45	< 0.1%

Most frequent character per script

Common

Value	Count	Frequency (%)
1	31335	13.9%
2	25694	11.4%
-	24941	11.1%
3	21555	9.6%
4	19380	8.6%
5	18309	8.1%
6	17448	7.8%
0	17150	7.6%
7	16984	7.5%
8	16613	7.4%

Latin

Value	Count	Frequency (%)
v	9	20.0%
a	9	20.0%
l	9	20.0%
u	9	20.0%
e	9	20.0%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	225145	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
1	31335	13.9%
2	25694	11.4%
-	24941	11.1%
3	21555	9.6%
4	19380	8.6%
5	18309	8.1%
6	17448	7.7%
0	17150	7.6%
7	16984	7.5%
8	16613	7.4%
Other values (6)	15736	7.0%

value-units
Categorical

HIGH CORRELATION

Distinct	2
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	390.8 KiB

megawatthours	50000
value-units	9

Length

Max length	13
Median length	13
Mean length	12.99964006
Min length	11

Characters and Unicode

Total characters	650099
Distinct characters	16
Distinct categories	2 ?
Distinct scripts	2 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	megawatthours
2nd row	megawatthours
3rd row	megawatthours
4th row	megawatthours
5th row	megawatthours

Common Values

Value	Count	Frequency (%)
megawatthours	50000	> 99.9%
value-units	9	< 0.1%

Length

Histogram of lengths of the category

Category Frequency Plot

Value	Count	Frequency (%)
megawatthours	50000	> 99.9%
value-units	9	< 0.1%

Most occurring characters

Value	Count	Frequency (%)
a	100009	15.4%
t	100009	15.4%
u	50018	7.7%
e	50009	7.7%
s	50009	7.7%
m	50000	7.7%
g	50000	7.7%
w	50000	7.7%
h	50000	7.7%
o	50000	7.7%
Other values (6)	50045	7.7%

Most occurring categories

Value	Count	Frequency (%)
Lowercase Letter	650090	> 99.9%
Dash Punctuation	9	< 0.1%

Most frequent character per category

Lowercase Letter

Value	Count	Frequency (%)
a	100009	15.4%
t	100009	15.4%
u	50018	7.7%
e	50009	7.7%
s	50009	7.7%
m	50000	7.7%
g	50000	7.7%
w	50000	7.7%
h	50000	7.7%
o	50000	7.7%
Other values (5)	50036	7.7%

Dash Punctuation

Value	Count	Frequency (%)
-	9	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Latin	650090	> 99.9%
Common	9	< 0.1%

Most frequent character per script

Latin

Value	Count	Frequency (%)
a	100009	15.4%
t	100009	15.4%
u	50018	7.7%
e	50009	7.7%
s	50009	7.7%
m	50000	7.7%
g	50000	7.7%
w	50000	7.7%
h	50000	7.7%
o	50000	7.7%
Other values (5)	50036	7.7%

Common

Value	Count	Frequency (%)
-	9	100.0%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	650099	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
a	100009	15.4%
t	100009	15.4%
u	50018	7.7%
e	50009	7.7%
s	50009	7.7%
m	50000	7.7%
g	50000	7.7%
w	50000	7.7%
h	50000	7.7%
o	50000	7.7%
Other values (6)	50045	7.7%

Unnamed: 0

Unnamed: 0

Auto

The auto setting is an easily interpretable pairwise column metric of the following mapping: vartype-vartype : method, categorical-categorical : Cramer's V, numerical-categorical : Cramer's V (using a discretized numerical column), numerical-numerical : Spearman's ρ. This configuration uses the best suitable for each pair of columns.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Cramér's V (φc)

Cramér's V is an association measure for nominal random variables. The coefficient ranges from 0 to 1, with 0 indicating independence and 1 indicating perfect association. The empirical estimators used for Cramér's V have been proved to be biased, even for large samples. We use a bias-corrected measure that has been proposed by Bergsma in 2013 that can be found here.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

The dendrogram allows you to more fully correlate variable completion, revealing trends deeper than the pairwise ones visible in the correlation heatmap.

First rows

	Unnamed: 0	period	fromba	fromba-name	toba	toba-name	timezone	value	value-units
0	0.0	2022-11-12	PACE	PacifiCorp East	LDWP	Los Angeles Department of Water and Power	Mountain	-1224	megawatthours
1	1.0	2022-11-12	SPA	Southwestern Power Administration	MISO	Midcontinent Independent System Operator, Inc.	Eastern	6259	megawatthours
2	2.0	2022-11-12	BPAT	Bonneville Power Administration	AVRN	Avangrid Renewables, LLC	Eastern	-143	megawatthours
3	3.0	2022-11-12	CISO	California Independent System Operator	LDWP	Los Angeles Department of Water and Power	Eastern	826	megawatthours
4	4.0	2022-11-12	TEN	Tennessee	CAR	Carolinas	Central	538	megawatthours
5	5.0	2022-11-12	PSEI	Puget Sound Energy, Inc.	TPWR	City of Tacoma, Department of Public Utilities, Light Division	Arizona	-1313	megawatthours
6	6.0	2022-11-12	TEC	Tampa Electric Company	SEC	Seminole Electric Cooperative	Eastern	-3341	megawatthours
7	7.0	2022-11-12	IID	Imperial Irrigation District	WALC	Western Area Power Administration - Desert Southwest Region	Central	423	megawatthours
8	8.0	2022-11-12	PACE	PacifiCorp East	NWMT	NorthWestern Corporation	Arizona	5824	megawatthours
9	9.0	2022-11-12	TIDC	Turlock Irrigation District	CISO	California Independent System Operator	Arizona	-7262	megawatthours

Last rows

	Unnamed: 0	period	fromba	fromba-name	toba	toba-name	timezone	value	value-units
49999	4990.0	2022-10-13	SC	South Carolina Public Service Authority	SOCO	Southern Company Services, Inc. - Trans	Arizona	-2615	megawatthours
50000	4991.0	2022-10-13	TEC	Tampa Electric Company	FPC	Duke Energy Florida, Inc.	Arizona	-9362	megawatthours
50001	4992.0	2022-10-13	FMPP	Florida Municipal Power Pool	FPC	Duke Energy Florida, Inc.	Arizona	10975	megawatthours
50002	4993.0	2022-10-13	SOCO	Southern Company Services, Inc. - Trans	FPC	Duke Energy Florida, Inc.	Arizona	61	megawatthours
50003	4994.0	2022-10-13	SWPP	Southwest Power Pool	SPA	Southwestern Power Administration	Pacific	250	megawatthours
50004	4995.0	2022-10-13	MIDA	Mid-Atlantic	MIDW	Midwest	Arizona	84871	megawatthours
50005	4996.0	2022-10-13	FPL	Florida Power & Light Co.	TEC	Tampa Electric Company	Pacific	-5727	megawatthours
50006	4997.0	2022-10-13	DUK	Duke Energy Carolinas	CPLW	Duke Energy Progress West	Pacific	3278	megawatthours
50007	4998.0	2022-10-13	NY	New York	NE	New England	Pacific	9168	megawatthours
50008	4999.0	2022-10-13	FPL	Florida Power & Light Co.	TEC	Tampa Electric Company	Central	-5857	megawatthours

Most frequently occurring

	Unnamed: 0	period	fromba	fromba-name	toba	toba-name	timezone	value	value-units	# duplicates
0	NaN	period	fromba	fromba-name	toba	toba-name	timezone	value	value-units	9

Overview

Variables

Common Values

Length

Most occurring characters

Most occurring categories

Most frequent character per category

Decimal Number

Lowercase Letter

Dash Punctuation

Most occurring scripts

Most frequent character per script

Common

Latin

Most occurring blocks

Most frequent character per block

ASCII

Common Values

Length

Most occurring characters

Most occurring categories

Most frequent character per category

Uppercase Letter

Lowercase Letter

Decimal Number

Most occurring scripts

Most frequent character per script

Latin

Common

Most occurring blocks

Most frequent character per block

ASCII

Common Values

Length

Most occurring characters

Most occurring categories

Most frequent character per category

Lowercase Letter

Uppercase Letter

Decimal Number

Other Punctuation

Space Separator

Dash Punctuation

Most occurring scripts

Most frequent character per script

Latin

Common

Most occurring blocks

Most frequent character per block

ASCII

Common Values

Length

Most occurring characters

Most occurring categories

Most frequent character per category

Uppercase Letter

Lowercase Letter

Most occurring scripts

Most frequent character per script

Latin

Most occurring blocks

Most frequent character per block

ASCII

Common Values

Length

Most occurring characters

Most occurring categories

Most frequent character per category

Lowercase Letter

Uppercase Letter

Other Punctuation

Decimal Number

Space Separator

Dash Punctuation

Most occurring scripts

Most frequent character per script

Latin

Common

Most occurring blocks

Most frequent character per block