day 13 ML practice problems
categorizing Iris
categorizing practice problem
We can use machine learning to classify the types of Iris
preparing the data
from sklearn.datasets import load_iris
iris = load_iris()
print(dir(iris))
# dir()는 객체가 어떤 변수와 메서드를 가지고 있는지 나열함
['DESCR', 'data', 'feature_names', 'filename', 'frame', 'target', 'target_names']
iris.keys()
dict_keys(['data', 'target', 'frame', 'target_names', 'DESCR', 'feature_names', 'filename'])
iris_data = iris.data
print(iris_data.shape)
#shape는 배열의 형상정보를 출력
#we can see taht 150 data has 4 info each
(150, 4)
iris_data[0]
array([5.1, 3.5, 1.4, 0.2])
iris_label = iris.target
print(iris_label.shape)
iris_label
(150,)
array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2])
iris.target_names
array(['setosa', 'versicolor', 'virginica'], dtype='<U10')
print(iris.DESCR)
.. _iris_dataset:
Iris plants dataset
--------------------
**Data Set Characteristics:**
:Number of Instances: 150 (50 in each of three classes)
:Number of Attributes: 4 numeric, predictive attributes and the class
:Attribute Information:
- sepal length in cm
- sepal width in cm
- petal length in cm
- petal width in cm
- class:
- Iris-Setosa
- Iris-Versicolour
- Iris-Virginica
:Summary Statistics:
============== ==== ==== ======= ===== ====================
Min Max Mean SD Class Correlation
============== ==== ==== ======= ===== ====================
sepal length: 4.3 7.9 5.84 0.83 0.7826
sepal width: 2.0 4.4 3.05 0.43 -0.4194
petal length: 1.0 6.9 3.76 1.76 0.9490 (high!)
petal width: 0.1 2.5 1.20 0.76 0.9565 (high!)
============== ==== ==== ======= ===== ====================
:Missing Attribute Values: None
:Class Distribution: 33.3% for each of 3 classes.
:Creator: R.A. Fisher
:Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)
:Date: July, 1988
The famous Iris database, first used by Sir R.A. Fisher. The dataset is taken
from Fisher's paper. Note that it's the same as in R, but not as in the UCI
Machine Learning Repository, which has two wrong data points.
This is perhaps the best known database to be found in the
pattern recognition literature. Fisher's paper is a classic in the field and
is referenced frequently to this day. (See Duda & Hart, for example.) The
data set contains 3 classes of 50 instances each, where each class refers to a
type of iris plant. One class is linearly separable from the other 2; the
latter are NOT linearly separable from each other.
.. topic:: References
- Fisher, R.A. "The use of multiple measurements in taxonomic problems"
Annual Eugenics, 7, Part II, 179-188 (1936); also in "Contributions to
Mathematical Statistics" (John Wiley, NY, 1950).
- Duda, R.O., & Hart, P.E. (1973) Pattern Classification and Scene Analysis.
(Q327.D83) John Wiley & Sons. ISBN 0-471-22361-1. See page 218.
- Dasarathy, B.V. (1980) "Nosing Around the Neighborhood: A New System
Structure and Classification Rule for Recognition in Partially Exposed
Environments". IEEE Transactions on Pattern Analysis and Machine
Intelligence, Vol. PAMI-2, No. 1, 67-71.
- Gates, G.W. (1972) "The Reduced Nearest Neighbor Rule". IEEE Transactions
on Information Theory, May 1972, 431-433.
- See also: 1988 MLC Proceedings, 54-64. Cheeseman et al"s AUTOCLASS II
conceptual clustering system finds 3 classes in the data.
- Many, many more ...
iris.feature_names
['sepal length (cm)',
'sepal width (cm)',
'petal length (cm)',
'petal width (cm)']
iris.filename
'C:\\Users\\jwl23\\anaconda3\\lib\\site-packages\\sklearn\\datasets\\data\\iris.csv'
creating the model
import pandas as pd
print(pd.__version__)
1.1.3
iris_df = pd.DataFrame(data=iris_data, columns=iris.feature_names)
iris_df
| sepal length (cm) | sepal width (cm) | petal length (cm) | petal width (cm) | |
|---|---|---|---|---|
| 0 | 5.1 | 3.5 | 1.4 | 0.2 |
| 1 | 4.9 | 3.0 | 1.4 | 0.2 |
| 2 | 4.7 | 3.2 | 1.3 | 0.2 |
| 3 | 4.6 | 3.1 | 1.5 | 0.2 |
| 4 | 5.0 | 3.6 | 1.4 | 0.2 |
| ... | ... | ... | ... | ... |
| 145 | 6.7 | 3.0 | 5.2 | 2.3 |
| 146 | 6.3 | 2.5 | 5.0 | 1.9 |
| 147 | 6.5 | 3.0 | 5.2 | 2.0 |
| 148 | 6.2 | 3.4 | 5.4 | 2.3 |
| 149 | 5.9 | 3.0 | 5.1 | 1.8 |
150 rows × 4 columns
iris_df["label"] = iris.target
iris_df
| sepal length (cm) | sepal width (cm) | petal length (cm) | petal width (cm) | label | |
|---|---|---|---|---|---|
| 0 | 5.1 | 3.5 | 1.4 | 0.2 | 0 |
| 1 | 4.9 | 3.0 | 1.4 | 0.2 | 0 |
| 2 | 4.7 | 3.2 | 1.3 | 0.2 | 0 |
| 3 | 4.6 | 3.1 | 1.5 | 0.2 | 0 |
| 4 | 5.0 | 3.6 | 1.4 | 0.2 | 0 |
| ... | ... | ... | ... | ... | ... |
| 145 | 6.7 | 3.0 | 5.2 | 2.3 | 2 |
| 146 | 6.3 | 2.5 | 5.0 | 1.9 | 2 |
| 147 | 6.5 | 3.0 | 5.2 | 2.0 | 2 |
| 148 | 6.2 | 3.4 | 5.4 | 2.3 | 2 |
| 149 | 5.9 | 3.0 | 5.1 | 1.8 | 2 |
150 rows × 5 columns
we added iris.target as label column in irs_df. The first four columns are the features. The last column label is the target.
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(iris_data,
iris_label,
test_size=0.2,
random_state=7)
print('X_train 개수: ', len(X_train), ', X_test 개수: ', len(X_test))
X_train 개수: 120 , X_test 개수: 30
X_train.shape, y_train.shape
((120, 4), (120,))
X_test.shape, y_test.shape
((30, 4), (30,))
y_train, y_test
(array([2, 1, 0, 2, 1, 0, 0, 0, 0, 2, 2, 1, 2, 2, 1, 0, 1, 1, 2, 0, 0, 0,
2, 0, 2, 1, 1, 1, 0, 0, 0, 1, 2, 1, 1, 0, 2, 0, 0, 2, 2, 0, 2, 0,
1, 2, 1, 0, 1, 0, 2, 2, 1, 0, 0, 1, 2, 0, 2, 2, 1, 0, 1, 0, 2, 2,
0, 0, 2, 1, 2, 2, 1, 0, 0, 2, 0, 0, 1, 2, 2, 1, 1, 0, 2, 0, 0, 1,
1, 2, 0, 1, 1, 2, 2, 1, 2, 0, 1, 1, 0, 0, 0, 1, 1, 0, 2, 2, 1, 2,
0, 2, 1, 1, 0, 2, 1, 2, 1, 0]),
array([2, 1, 0, 1, 2, 0, 1, 1, 0, 1, 1, 1, 0, 2, 0, 1, 2, 2, 0, 0, 1, 2,
1, 2, 2, 2, 1, 1, 2, 2]))
training the model
classifying types of iris is a supervised learning because there is present answers. It is a classification problem.
We will be using the decision tree method to classify the data.
from sklearn.tree import DecisionTreeClassifier
decision_tree = DecisionTreeClassifier(random_state=32)
print(decision_tree._estimator_type)
classifier
decision_tree.fit(X_train, y_train)
DecisionTreeClassifier(random_state=32)
evaluating the model
X_test data does not have label but only features. Therefore, the finished return of decision tree prediction is y_pred
y_pred = decision_tree.predict(X_test)
y_pred
array([2, 1, 0, 1, 2, 0, 1, 1, 0, 1, 2, 1, 0, 2, 0, 2, 2, 2, 0, 0, 1, 2,
1, 1, 2, 2, 1, 1, 2, 2])
we got 30 data predictions. We can compare to this to y_test
y_test
array([2, 1, 0, 1, 2, 0, 1, 1, 0, 1, 1, 1, 0, 2, 0, 1, 2, 2, 0, 0, 1, 2,
1, 2, 2, 2, 1, 1, 2, 2])
from sklearn.metrics import accuracy_score
accuracy = accuracy_score(y_test, y_pred)
accuracy
0.9
since we predicted 30 data, the equation is 30*0.9 = 27.
In other words, out of 30, 27 were correctly classified while 3 were incorrectly classified.
other models
# (1) 필요한 모듈 import
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import classification_report
# (2) 데이터 준비
iris = load_iris()
iris_data = iris.data
iris_label = iris.target
# (3) train, test 데이터 분리
X_train, X_test, y_train, y_test = train_test_split(iris_data,
iris_label,
test_size=0.2,
random_state=7)
# (4) 모델 학습 및 예측
decision_tree = DecisionTreeClassifier(random_state=32)
decision_tree.fit(X_train, y_train)
y_pred = decision_tree.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 1.00 1.00 7
1 0.91 0.83 0.87 12
2 0.83 0.91 0.87 11
accuracy 0.90 30
macro avg 0.91 0.91 0.91 30
weighted avg 0.90 0.90 0.90 30
Above is the lines of codes we ran for previously.
From this, we can change the step (4) to use different models.
Random Forest
- We will try using Random Forest which is a collection of many decision trees selected at random
from sklearn.ensemble import RandomForestClassifier
X_train, X_test, y_train, y_test = train_test_split(iris_data,
iris_label,
test_size=0.2,
random_state=21)
random_forest = RandomForestClassifier(random_state=32)
random_forest.fit(X_train, y_train)
y_pred = random_forest.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 1.00 1.00 11
1 1.00 0.83 0.91 12
2 0.78 1.00 0.88 7
accuracy 0.93 30
macro avg 0.93 0.94 0.93 30
weighted avg 0.95 0.93 0.93 30
Support Vector Machine (SVM)
from sklearn import svm
svm_model = svm.SVC()
print(svm_model._estimator_type)
classifier
X_train, X_test, y_train, y_test = train_test_split(iris_data,
iris_label,
test_size=0.2,
random_state=21)
svm_model.fit(X_train, y_train)
y_pred = svm_model.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 1.00 1.00 11
1 0.91 0.83 0.87 12
2 0.75 0.86 0.80 7
accuracy 0.90 30
macro avg 0.89 0.90 0.89 30
weighted avg 0.91 0.90 0.90 30
Stochastic Gradient Descent Classifier (SGDClassifier)
from sklearn.linear_model import SGDClassifier
sgd_model = SGDClassifier()
print(sgd_model._estimator_type)
classifier
X_train, X_test, y_train, y_test = train_test_split(iris_data,
iris_label,
test_size=0.2,
random_state=21)
sgd_model.fit(X_train, y_train)
y_pred = sgd_model.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 1.00 1.00 11
1 1.00 0.67 0.80 12
2 0.64 1.00 0.78 7
accuracy 0.87 30
macro avg 0.88 0.89 0.86 30
weighted avg 0.92 0.87 0.87 30
Logistic Regression
from sklearn.linear_model import LogisticRegression
logistic_model = LogisticRegression()
print(logistic_model._estimator_type)
classifier
X_train, X_test, y_train, y_test = train_test_split(iris_data,
iris_label,
test_size=0.2,
random_state=21)
logistic_model.fit(X_train, y_train)
y_pred = logistic_model.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 1.00 1.00 11
1 1.00 0.83 0.91 12
2 0.78 1.00 0.88 7
accuracy 0.93 30
macro avg 0.93 0.94 0.93 30
weighted avg 0.95 0.93 0.93 30
testing the model
error in accuracy
we will see what the errors in testing for accuracy is by looking at the handwriting dataset MNIST
from sklearn.datasets import load_digits
digits = load_digits()
digits.keys()
dict_keys(['data', 'target', 'frame', 'feature_names', 'target_names', 'images', 'DESCR'])
digits_data = digits.data
digits_data.shape
(1797, 64)
digits_data[0]
array([ 0., 0., 5., 13., 9., 1., 0., 0., 0., 0., 13., 15., 10.,
15., 5., 0., 0., 3., 15., 2., 0., 11., 8., 0., 0., 4.,
12., 0., 0., 8., 8., 0., 0., 5., 8., 0., 0., 9., 8.,
0., 0., 4., 11., 0., 1., 12., 7., 0., 0., 2., 14., 5.,
10., 12., 0., 0., 0., 0., 6., 13., 10., 0., 0., 0.])
import matplotlib.pyplot as plt
%matplotlib inline
plt.imshow(digits.data[0].reshape(8, 8), cmap='gray')
plt.axis('off')
plt.show()
for i in range(10):
plt.subplot(2, 5, i+1)
plt.imshow(digits.data[i].reshape(8, 8), cmap='gray')
plt.axis('off')
plt.show()
digits_label = digits.target
print(digits_label.shape)
digits_label[:20]
(1797,)
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
We can solve this problem by using classification. However, we will purposefully change it to checking if the data is number 3 or not. We need to chagne traget digital_label. If it is not 3, it will be 0
new_label = [3 if i == 3 else 0 for i in digits_label]
new_label[:20]
[0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0]
from sklearn.metrics import accuracy_score
X_train, X_test, y_train, y_test = train_test_split(digits_data,
new_label,
test_size=0.2,
random_state=15)
decision_tree = DecisionTreeClassifier(random_state=15)
decision_tree.fit(X_train, y_train)
y_pred = decision_tree.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)
accuracy
0.9388888888888889
we got a return of 93% accuracy which is high. However we neglect that by changing the dataset, we have made the dataset very unbalanced. label has many 0 but has very little 3. 90% of all label is 0. This means that even if we dont train the model, if the model chooses 0 as the answer, the accuracy will result in 90%.
we can test for this by creating an list same length as y_pred but with all 0.
fake_pred = [0] * len(y_pred)
accuracy = accuracy_score(y_test, fake_pred)
accuracy
0.925
answers and incorrect answers
accuracy only concerns the correct data predictions. However, it is also important to look at incorrect predictions as well.
we can look to the confusion matrix for help
the confusion matrix includes …
-
TP(True Positive) : 실제 환자에게 양성판정 (참 양성)
-
FN(False Negative) : 실제 환자에게 음성판정 (거짓 음성)
-
FP(False Positive) : 건강한 사람에게 양성판정 (거짓 양성)
-
TN(True Negative) : 건강한 사람에게 음성판정 (참 음성)
정밀도(Precision), 재현율(Recall, Sensitivity), F1 스코어(f1 score)
For example cases,
in classifying spam emails, precision is more important because we should not classify normal emails as spam.
in cancer patient diagnosis, doctors can risk missing a single patient. So the recall value becomes more important
from sklearn.metrics import confusion_matrix
confusion_matrix(y_test, y_pred)
array([[320, 13],
[ 9, 18]], dtype=int64)
confusion_matrix(y_test, fake_pred)
array([[333, 0],
[ 27, 0]], dtype=int64)
from sklearn.metrics import classification_report
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 0.97 0.96 0.97 333
3 0.58 0.67 0.62 27
accuracy 0.94 360
macro avg 0.78 0.81 0.79 360
weighted avg 0.94 0.94 0.94 360
print(classification_report(y_test, fake_pred))
precision recall f1-score support
0 0.93 1.00 0.96 333
3 0.00 0.00 0.00 27
accuracy 0.93 360
macro avg 0.46 0.50 0.48 360
weighted avg 0.86 0.93 0.89 360
C:\Users\jwl23\anaconda3\lib\site-packages\sklearn\metrics\_classification.py:1221: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
_warn_prf(average, modifier, msg_start, len(result))
when we predict for 0, the model does great but when predicting for 3, it failed in getting any correct answer
accuracy_score(y_test, y_pred), accuracy_score(y_test, fake_pred)
(0.9388888888888889, 0.925)
there is barely any difference between the y_pred and fake_pred.
Therefore we must be mindful of the distribution in data in label and look to the confusion matrix to check if the model is good.
practice problem
손글씨 분류
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
# (1) 필요한 모듈 import
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import classification_report
# (2) 데이터 준비
digits = load_digits()
digits_data = digits.data
digits_label = digits.target
digits.keys()
dict_keys(['data', 'target', 'frame', 'feature_names', 'target_names', 'images', 'DESCR'])
digits["target_names"]
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
print(digits["DESCR"])
.. _digits_dataset:
Optical recognition of handwritten digits dataset
--------------------------------------------------
**Data Set Characteristics:**
:Number of Instances: 5620
:Number of Attributes: 64
:Attribute Information: 8x8 image of integer pixels in the range 0..16.
:Missing Attribute Values: None
:Creator: E. Alpaydin (alpaydin '@' boun.edu.tr)
:Date: July; 1998
This is a copy of the test set of the UCI ML hand-written digits datasets
https://archive.ics.uci.edu/ml/datasets/Optical+Recognition+of+Handwritten+Digits
The data set contains images of hand-written digits: 10 classes where
each class refers to a digit.
Preprocessing programs made available by NIST were used to extract
normalized bitmaps of handwritten digits from a preprinted form. From a
total of 43 people, 30 contributed to the training set and different 13
to the test set. 32x32 bitmaps are divided into nonoverlapping blocks of
4x4 and the number of on pixels are counted in each block. This generates
an input matrix of 8x8 where each element is an integer in the range
0..16. This reduces dimensionality and gives invariance to small
distortions.
For info on NIST preprocessing routines, see M. D. Garris, J. L. Blue, G.
T. Candela, D. L. Dimmick, J. Geist, P. J. Grother, S. A. Janet, and C.
L. Wilson, NIST Form-Based Handprint Recognition System, NISTIR 5469,
1994.
.. topic:: References
- C. Kaynak (1995) Methods of Combining Multiple Classifiers and Their
Applications to Handwritten Digit Recognition, MSc Thesis, Institute of
Graduate Studies in Science and Engineering, Bogazici University.
- E. Alpaydin, C. Kaynak (1998) Cascading Classifiers, Kybernetika.
- Ken Tang and Ponnuthurai N. Suganthan and Xi Yao and A. Kai Qin.
Linear dimensionalityreduction using relevance weighted LDA. School of
Electrical and Electronic Engineering Nanyang Technological University.
2005.
- Claudio Gentile. A New Approximate Maximal Margin Classification
Algorithm. NIPS. 2000.
# (3) train, test 데이터 분리
X_train, X_test, y_train, y_test = train_test_split(digits_data,
digits_label,
test_size=0.2,
random_state=7)
# (4) 모델 학습 및 예측 Decision Tree 사용해 보기
decision_tree = DecisionTreeClassifier(random_state=32)
decision_tree.fit(X_train, y_train)
y_pred = decision_tree.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 0.98 0.99 43
1 0.81 0.81 0.81 42
2 0.79 0.82 0.80 40
3 0.79 0.91 0.85 34
4 0.83 0.95 0.89 37
5 0.90 0.96 0.93 28
6 0.84 0.93 0.88 28
7 0.96 0.82 0.89 33
8 0.88 0.65 0.75 43
9 0.78 0.78 0.78 32
accuracy 0.86 360
macro avg 0.86 0.86 0.86 360
weighted avg 0.86 0.86 0.85 360
# (4) 모델 학습 및 예측 Random Forest
from sklearn.ensemble import RandomForestClassifier
random_forest = RandomForestClassifier(random_state=32)
random_forest.fit(X_train, y_train)
y_pred = random_forest.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 0.98 0.99 43
1 0.93 1.00 0.97 42
2 1.00 1.00 1.00 40
3 1.00 1.00 1.00 34
4 0.93 1.00 0.96 37
5 0.90 0.96 0.93 28
6 1.00 0.96 0.98 28
7 0.94 0.97 0.96 33
8 1.00 0.84 0.91 43
9 0.94 0.94 0.94 32
accuracy 0.96 360
macro avg 0.96 0.96 0.96 360
weighted avg 0.97 0.96 0.96 360
# (4) 모델 학습 및 예측 svm
from sklearn import svm
svm_model = svm.SVC()
svm_model.fit(X_train, y_train)
y_pred = svm_model.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 1.00 1.00 43
1 0.95 1.00 0.98 42
2 1.00 1.00 1.00 40
3 1.00 1.00 1.00 34
4 1.00 1.00 1.00 37
5 0.93 1.00 0.97 28
6 1.00 1.00 1.00 28
7 1.00 1.00 1.00 33
8 1.00 0.93 0.96 43
9 1.00 0.97 0.98 32
accuracy 0.99 360
macro avg 0.99 0.99 0.99 360
weighted avg 0.99 0.99 0.99 360
# (4) 모델 학습 및 예측 SGDClassifier
from sklearn.linear_model import SGDClassifier
sgd_model = SGDClassifier()
svm_model.fit(X_train, y_train)
y_pred = svm_model.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 1.00 1.00 43
1 0.95 1.00 0.98 42
2 1.00 1.00 1.00 40
3 1.00 1.00 1.00 34
4 1.00 1.00 1.00 37
5 0.93 1.00 0.97 28
6 1.00 1.00 1.00 28
7 1.00 1.00 1.00 33
8 1.00 0.93 0.96 43
9 1.00 0.97 0.98 32
accuracy 0.99 360
macro avg 0.99 0.99 0.99 360
weighted avg 0.99 0.99 0.99 360
# (4) 모델 학습 및 예측 SGDClassifier
from sklearn.linear_model import LogisticRegression
logistic_model = LogisticRegression()
logistic_model.fit(X_train, y_train)
y_pred = logistic_model.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 1.00 1.00 43
1 0.95 0.95 0.95 42
2 0.98 1.00 0.99 40
3 0.94 0.97 0.96 34
4 0.97 1.00 0.99 37
5 0.82 0.96 0.89 28
6 1.00 0.96 0.98 28
7 0.97 0.97 0.97 33
8 0.92 0.81 0.86 43
9 0.97 0.91 0.94 32
accuracy 0.95 360
macro avg 0.95 0.95 0.95 360
weighted avg 0.95 0.95 0.95 360
C:\Users\jwl23\anaconda3\lib\site-packages\sklearn\linear_model\_logistic.py:762: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
classifying handwritten numbers requires precision more than other indicators because falsely classifying a number would cause the entire line of number to be incorrect.
와인 분류
from sklearn.datasets import load_wine
wine = load_wine()
wine_data = wine.data
wine_label = wine.target
wine.keys()
dict_keys(['data', 'target', 'frame', 'target_names', 'DESCR', 'feature_names'])
wine.target_names
array(['class_0', 'class_1', 'class_2'], dtype='<U7')
print(wine.DESCR)
.. _wine_dataset:
Wine recognition dataset
------------------------
**Data Set Characteristics:**
:Number of Instances: 178 (50 in each of three classes)
:Number of Attributes: 13 numeric, predictive attributes and the class
:Attribute Information:
- Alcohol
- Malic acid
- Ash
- Alcalinity of ash
- Magnesium
- Total phenols
- Flavanoids
- Nonflavanoid phenols
- Proanthocyanins
- Color intensity
- Hue
- OD280/OD315 of diluted wines
- Proline
- class:
- class_0
- class_1
- class_2
:Summary Statistics:
============================= ==== ===== ======= =====
Min Max Mean SD
============================= ==== ===== ======= =====
Alcohol: 11.0 14.8 13.0 0.8
Malic Acid: 0.74 5.80 2.34 1.12
Ash: 1.36 3.23 2.36 0.27
Alcalinity of Ash: 10.6 30.0 19.5 3.3
Magnesium: 70.0 162.0 99.7 14.3
Total Phenols: 0.98 3.88 2.29 0.63
Flavanoids: 0.34 5.08 2.03 1.00
Nonflavanoid Phenols: 0.13 0.66 0.36 0.12
Proanthocyanins: 0.41 3.58 1.59 0.57
Colour Intensity: 1.3 13.0 5.1 2.3
Hue: 0.48 1.71 0.96 0.23
OD280/OD315 of diluted wines: 1.27 4.00 2.61 0.71
Proline: 278 1680 746 315
============================= ==== ===== ======= =====
:Missing Attribute Values: None
:Class Distribution: class_0 (59), class_1 (71), class_2 (48)
:Creator: R.A. Fisher
:Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)
:Date: July, 1988
This is a copy of UCI ML Wine recognition datasets.
https://archive.ics.uci.edu/ml/machine-learning-databases/wine/wine.data
The data is the results of a chemical analysis of wines grown in the same
region in Italy by three different cultivators. There are thirteen different
measurements taken for different constituents found in the three types of
wine.
Original Owners:
Forina, M. et al, PARVUS -
An Extendible Package for Data Exploration, Classification and Correlation.
Institute of Pharmaceutical and Food Analysis and Technologies,
Via Brigata Salerno, 16147 Genoa, Italy.
Citation:
Lichman, M. (2013). UCI Machine Learning Repository
[https://archive.ics.uci.edu/ml]. Irvine, CA: University of California,
School of Information and Computer Science.
.. topic:: References
(1) S. Aeberhard, D. Coomans and O. de Vel,
Comparison of Classifiers in High Dimensional Settings,
Tech. Rep. no. 92-02, (1992), Dept. of Computer Science and Dept. of
Mathematics and Statistics, James Cook University of North Queensland.
(Also submitted to Technometrics).
The data was used with many others for comparing various
classifiers. The classes are separable, though only RDA
has achieved 100% correct classification.
(RDA : 100%, QDA 99.4%, LDA 98.9%, 1NN 96.1% (z-transformed data))
(All results using the leave-one-out technique)
(2) S. Aeberhard, D. Coomans and O. de Vel,
"THE CLASSIFICATION PERFORMANCE OF RDA"
Tech. Rep. no. 92-01, (1992), Dept. of Computer Science and Dept. of
Mathematics and Statistics, James Cook University of North Queensland.
(Also submitted to Journal of Chemometrics).
# (3) train, test 데이터 분리
X_train, X_test, y_train, y_test = train_test_split(wine_data,
wine_label,
test_size=0.2,
random_state=7)
# (4) 모델 학습 및 예측 Decision Tree 사용해 보기
decision_tree = DecisionTreeClassifier(random_state=32)
decision_tree.fit(X_train, y_train)
y_pred = decision_tree.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 1.00 1.00 7
1 0.89 1.00 0.94 17
2 1.00 0.83 0.91 12
accuracy 0.94 36
macro avg 0.96 0.94 0.95 36
weighted avg 0.95 0.94 0.94 36
# (4) 모델 학습 및 예측 Random Forest
from sklearn.ensemble import RandomForestClassifier
random_forest = RandomForestClassifier(random_state=32)
random_forest.fit(X_train, y_train)
y_pred = random_forest.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 1.00 1.00 7
1 1.00 1.00 1.00 17
2 1.00 1.00 1.00 12
accuracy 1.00 36
macro avg 1.00 1.00 1.00 36
weighted avg 1.00 1.00 1.00 36
# (4) 모델 학습 및 예측 svm
from sklearn import svm
svm_model = svm.SVC()
svm_model.fit(X_train, y_train)
y_pred = svm_model.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 0.86 0.86 0.86 7
1 0.58 0.88 0.70 17
2 0.33 0.08 0.13 12
accuracy 0.61 36
macro avg 0.59 0.61 0.56 36
weighted avg 0.55 0.61 0.54 36
# (4) 모델 학습 및 예측 SGDClassifier
from sklearn.linear_model import SGDClassifier
sgd_model = SGDClassifier()
svm_model.fit(X_train, y_train)
y_pred = svm_model.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 0.86 0.86 0.86 7
1 0.58 0.88 0.70 17
2 0.33 0.08 0.13 12
accuracy 0.61 36
macro avg 0.59 0.61 0.56 36
weighted avg 0.55 0.61 0.54 36
# (4) 모델 학습 및 예측 SGDClassifier
from sklearn.linear_model import LogisticRegression
logistic_model = LogisticRegression()
logistic_model.fit(X_train, y_train)
y_pred = logistic_model.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 0.86 0.92 7
1 0.94 1.00 0.97 17
2 1.00 1.00 1.00 12
accuracy 0.97 36
macro avg 0.98 0.95 0.96 36
weighted avg 0.97 0.97 0.97 36
C:\Users\jwl23\anaconda3\lib\site-packages\sklearn\linear_model\_logistic.py:762: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
for wine, same with hand written numbers, precision becomes more important because we are trying to classify wines by different cultivators. Correct classification becomes more important for these cultivators wanting their wine to be unique.
유방암 분류
from sklearn.datasets import load_breast_cancer
breast_cancer = load_breast_cancer()
breast_cancer_data = breast_cancer.data
breast_cancer_label = breast_cancer.target
breast_cancer.keys()
dict_keys(['data', 'target', 'frame', 'target_names', 'DESCR', 'feature_names', 'filename'])
breast_cancer.target_names
array(['malignant', 'benign'], dtype='<U9')
print(breast_cancer.DESCR)
.. _breast_cancer_dataset:
Breast cancer wisconsin (diagnostic) dataset
--------------------------------------------
**Data Set Characteristics:**
:Number of Instances: 569
:Number of Attributes: 30 numeric, predictive attributes and the class
:Attribute Information:
- radius (mean of distances from center to points on the perimeter)
- texture (standard deviation of gray-scale values)
- perimeter
- area
- smoothness (local variation in radius lengths)
- compactness (perimeter^2 / area - 1.0)
- concavity (severity of concave portions of the contour)
- concave points (number of concave portions of the contour)
- symmetry
- fractal dimension ("coastline approximation" - 1)
The mean, standard error, and "worst" or largest (mean of the three
worst/largest values) of these features were computed for each image,
resulting in 30 features. For instance, field 0 is Mean Radius, field
10 is Radius SE, field 20 is Worst Radius.
- class:
- WDBC-Malignant
- WDBC-Benign
:Summary Statistics:
===================================== ====== ======
Min Max
===================================== ====== ======
radius (mean): 6.981 28.11
texture (mean): 9.71 39.28
perimeter (mean): 43.79 188.5
area (mean): 143.5 2501.0
smoothness (mean): 0.053 0.163
compactness (mean): 0.019 0.345
concavity (mean): 0.0 0.427
concave points (mean): 0.0 0.201
symmetry (mean): 0.106 0.304
fractal dimension (mean): 0.05 0.097
radius (standard error): 0.112 2.873
texture (standard error): 0.36 4.885
perimeter (standard error): 0.757 21.98
area (standard error): 6.802 542.2
smoothness (standard error): 0.002 0.031
compactness (standard error): 0.002 0.135
concavity (standard error): 0.0 0.396
concave points (standard error): 0.0 0.053
symmetry (standard error): 0.008 0.079
fractal dimension (standard error): 0.001 0.03
radius (worst): 7.93 36.04
texture (worst): 12.02 49.54
perimeter (worst): 50.41 251.2
area (worst): 185.2 4254.0
smoothness (worst): 0.071 0.223
compactness (worst): 0.027 1.058
concavity (worst): 0.0 1.252
concave points (worst): 0.0 0.291
symmetry (worst): 0.156 0.664
fractal dimension (worst): 0.055 0.208
===================================== ====== ======
:Missing Attribute Values: None
:Class Distribution: 212 - Malignant, 357 - Benign
:Creator: Dr. William H. Wolberg, W. Nick Street, Olvi L. Mangasarian
:Donor: Nick Street
:Date: November, 1995
This is a copy of UCI ML Breast Cancer Wisconsin (Diagnostic) datasets.
https://goo.gl/U2Uwz2
Features are computed from a digitized image of a fine needle
aspirate (FNA) of a breast mass. They describe
characteristics of the cell nuclei present in the image.
Separating plane described above was obtained using
Multisurface Method-Tree (MSM-T) [K. P. Bennett, "Decision Tree
Construction Via Linear Programming." Proceedings of the 4th
Midwest Artificial Intelligence and Cognitive Science Society,
pp. 97-101, 1992], a classification method which uses linear
programming to construct a decision tree. Relevant features
were selected using an exhaustive search in the space of 1-4
features and 1-3 separating planes.
The actual linear program used to obtain the separating plane
in the 3-dimensional space is that described in:
[K. P. Bennett and O. L. Mangasarian: "Robust Linear
Programming Discrimination of Two Linearly Inseparable Sets",
Optimization Methods and Software 1, 1992, 23-34].
This database is also available through the UW CS ftp server:
ftp ftp.cs.wisc.edu
cd math-prog/cpo-dataset/machine-learn/WDBC/
.. topic:: References
- W.N. Street, W.H. Wolberg and O.L. Mangasarian. Nuclear feature extraction
for breast tumor diagnosis. IS&T/SPIE 1993 International Symposium on
Electronic Imaging: Science and Technology, volume 1905, pages 861-870,
San Jose, CA, 1993.
- O.L. Mangasarian, W.N. Street and W.H. Wolberg. Breast cancer diagnosis and
prognosis via linear programming. Operations Research, 43(4), pages 570-577,
July-August 1995.
- W.H. Wolberg, W.N. Street, and O.L. Mangasarian. Machine learning techniques
to diagnose breast cancer from fine-needle aspirates. Cancer Letters 77 (1994)
163-171.
# (3) train, test 데이터 분리
X_train, X_test, y_train, y_test = train_test_split(breast_cancer_data,
breast_cancer_label,
test_size=0.2,
random_state=7)
# (4) 모델 학습 및 예측 Decision Tree 사용해 보기
decision_tree = DecisionTreeClassifier(random_state=32)
decision_tree.fit(X_train, y_train)
y_pred = decision_tree.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 0.92 0.82 0.87 40
1 0.91 0.96 0.93 74
accuracy 0.91 114
macro avg 0.91 0.89 0.90 114
weighted avg 0.91 0.91 0.91 114
# (4) 모델 학습 및 예측 Random Forest
from sklearn.ensemble import RandomForestClassifier
random_forest = RandomForestClassifier(random_state=32)
random_forest.fit(X_train, y_train)
y_pred = random_forest.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 1.00 1.00 40
1 1.00 1.00 1.00 74
accuracy 1.00 114
macro avg 1.00 1.00 1.00 114
weighted avg 1.00 1.00 1.00 114
# (4) 모델 학습 및 예측 svm
from sklearn import svm
svm_model = svm.SVC()
svm_model.fit(X_train, y_train)
y_pred = svm_model.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 0.72 0.84 40
1 0.87 1.00 0.93 74
accuracy 0.90 114
macro avg 0.94 0.86 0.89 114
weighted avg 0.92 0.90 0.90 114
# (4) 모델 학습 및 예측 SGDClassifier
from sklearn.linear_model import SGDClassifier
sgd_model = SGDClassifier()
svm_model.fit(X_train, y_train)
y_pred = svm_model.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 0.72 0.84 40
1 0.87 1.00 0.93 74
accuracy 0.90 114
macro avg 0.94 0.86 0.89 114
weighted avg 0.92 0.90 0.90 114
# (4) 모델 학습 및 예측 SGDClassifier
from sklearn.linear_model import LogisticRegression
logistic_model = LogisticRegression()
logistic_model.fit(X_train, y_train)
y_pred = logistic_model.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 0.82 0.90 40
1 0.91 1.00 0.95 74
accuracy 0.94 114
macro avg 0.96 0.91 0.93 114
weighted avg 0.94 0.94 0.94 114
C:\Users\jwl23\anaconda3\lib\site-packages\sklearn\linear_model\_logistic.py:762: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
For cancer diagnosis, it is important for doctors to not miss a diagnosis for cancer. it is better to test for a negative rather than miss the chance to test for cancer at all. Threfore, recall becomes more important
Leave a comment