Econ 425T
Source (structured data): https://keras.io/examples/structured_data/structured_data_classification_from_scratch/.
Source (KerasTuner): https://keras.io/guides/keras_tuner/getting_started/
Display system information for reproducibility.
import IPython
print(IPython.sys_info()){'commit_hash': 'add5877a4',
'commit_source': 'installation',
'default_encoding': 'utf-8',
'ipython_path': '/Users/huazhou/opt/anaconda3/lib/python3.9/site-packages/IPython',
'ipython_version': '8.8.0',
'os_name': 'posix',
'platform': 'macOS-10.16-x86_64-i386-64bit',
'sys_executable': '/Users/huazhou/opt/anaconda3/bin/python3',
'sys_platform': 'darwin',
'sys_version': '3.9.12 (main, Apr 5 2022, 01:56:13) \n[Clang 12.0.0 ]'}
We illustrate the typical machine learning workflow for multi-layer perceptron (MLP) using the Heart data set from R ISLR2 package.
Initial splitting to test and non-test sets.
Pre-processing of data: not much is needed for regression trees.
Tune the cost complexity pruning hyper-parameter(s) using 10-fold cross-validation (CV) on the non-test data.
Choose the best model by validation.
Final prediction on the test data.
The goal is to predict the binary outcome AHD (Yes or No) of patients.
# Load the pandas library
import pandas as pd
# Load numpy for array manipulation
import numpy as np
# Load seaborn plotting library
import seaborn as sns
import matplotlib.pyplot as plt
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
# Set font sizes in plots
sns.set(font_scale = 1.2)
# Display all columns
pd.set_option('display.max_columns', None)
# Drop rows with NaNs (not consistent as other workflows!!!)
Heart = pd.read_csv("../../data/Heart.csv").drop(['Unnamed: 0'], axis = 1).dropna()
Heart['AHD'] = Heart['AHD'] == 'Yes'
Heart Age Sex ChestPain RestBP Chol Fbs RestECG MaxHR ExAng \
0 63 1 typical 145 233 1 2 150 0
1 67 1 asymptomatic 160 286 0 2 108 1
2 67 1 asymptomatic 120 229 0 2 129 1
3 37 1 nonanginal 130 250 0 0 187 0
4 41 0 nontypical 130 204 0 2 172 0
.. ... ... ... ... ... ... ... ... ...
297 57 0 asymptomatic 140 241 0 0 123 1
298 45 1 typical 110 264 0 0 132 0
299 68 1 asymptomatic 144 193 1 0 141 0
300 57 1 asymptomatic 130 131 0 0 115 1
301 57 0 nontypical 130 236 0 2 174 0
Oldpeak Slope Ca Thal AHD
0 2.3 3 0.0 fixed False
1 1.5 2 3.0 normal True
2 2.6 2 2.0 reversable True
3 3.5 3 0.0 normal False
4 1.4 1 0.0 normal False
.. ... ... ... ... ...
297 0.2 2 0.0 reversable True
298 1.2 2 0.0 reversable True
299 3.4 2 2.0 reversable True
300 1.2 2 1.0 reversable True
301 0.0 2 1.0 normal True
[297 rows x 14 columns]# Numerical summaries
Heart.describe(include = 'all') Age Sex ChestPain RestBP Chol \
count 297.000000 297.000000 297 297.000000 297.000000
unique NaN NaN 4 NaN NaN
top NaN NaN asymptomatic NaN NaN
freq NaN NaN 142 NaN NaN
mean 54.542088 0.676768 NaN 131.693603 247.350168
std 9.049736 0.468500 NaN 17.762806 51.997583
min 29.000000 0.000000 NaN 94.000000 126.000000
25% 48.000000 0.000000 NaN 120.000000 211.000000
50% 56.000000 1.000000 NaN 130.000000 243.000000
75% 61.000000 1.000000 NaN 140.000000 276.000000
max 77.000000 1.000000 NaN 200.000000 564.000000
Fbs RestECG MaxHR ExAng Oldpeak \
count 297.000000 297.000000 297.000000 297.000000 297.000000
unique NaN NaN NaN NaN NaN
top NaN NaN NaN NaN NaN
freq NaN NaN NaN NaN NaN
mean 0.144781 0.996633 149.599327 0.326599 1.055556
std 0.352474 0.994914 22.941562 0.469761 1.166123
min 0.000000 0.000000 71.000000 0.000000 0.000000
25% 0.000000 0.000000 133.000000 0.000000 0.000000
50% 0.000000 1.000000 153.000000 0.000000 0.800000
75% 0.000000 2.000000 166.000000 1.000000 1.600000
max 1.000000 2.000000 202.000000 1.000000 6.200000
Slope Ca Thal AHD
count 297.000000 297.000000 297 297
unique NaN NaN 3 2
top NaN NaN normal False
freq NaN NaN 164 160
mean 1.602694 0.676768 NaN NaN
std 0.618187 0.938965 NaN NaN
min 1.000000 0.000000 NaN NaN
25% 1.000000 0.000000 NaN NaN
50% 2.000000 0.000000 NaN NaN
75% 2.000000 1.000000 NaN NaN
max 3.000000 3.000000 NaN NaN Graphical summary:
# Graphical summaries
plt.figure()
sns.pairplot(data = Heart);
plt.show()We randomly split the data into 25% test data and 75% non-test data. Stratify on AHD.
For the non-test data, we further split into 80% training data and 20% validation data.
from sklearn.model_selection import train_test_split
# Initial test, non-test split
Heart_other, Heart_test = train_test_split(
Heart,
train_size = 0.75,
random_state = 425, # seed
stratify = Heart.AHD
)
# Train, validation split
Heart_train, Heart_val = train_test_split(
Heart_other,
train_size = 0.8,
random_state = 425, # seed
stratify = Heart_other.AHD
)
Heart_test.shape(75, 14)Heart_train.shape(177, 14)Heart_val.shape(45, 14)Let’s generate tf.data.Dataset objects for each dataframe:
def dataframe_to_dataset(dataframe):
dataframe = dataframe.copy()
labels = dataframe.pop("AHD")
ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels))
ds = ds.shuffle(buffer_size = len(dataframe))
return ds
train_ds = dataframe_to_dataset(Heart_train)
val_ds = dataframe_to_dataset(Heart_val)
test_ds = dataframe_to_dataset(Heart_test)Each Dataset yields a tuple (input, target) where input is a dictionary of features and target is the value 0 or 1:
for x, y in train_ds.take(1):
print("Input:", x)
print("Target:", y)Input: {'Age': <tf.Tensor: shape=(), dtype=int64, numpy=52>, 'Sex': <tf.Tensor: shape=(), dtype=int64, numpy=1>, 'ChestPain': <tf.Tensor: shape=(), dtype=string, numpy=b'asymptomatic'>, 'RestBP': <tf.Tensor: shape=(), dtype=int64, numpy=128>, 'Chol': <tf.Tensor: shape=(), dtype=int64, numpy=255>, 'Fbs': <tf.Tensor: shape=(), dtype=int64, numpy=0>, 'RestECG': <tf.Tensor: shape=(), dtype=int64, numpy=0>, 'MaxHR': <tf.Tensor: shape=(), dtype=int64, numpy=161>, 'ExAng': <tf.Tensor: shape=(), dtype=int64, numpy=1>, 'Oldpeak': <tf.Tensor: shape=(), dtype=float64, numpy=0.0>, 'Slope': <tf.Tensor: shape=(), dtype=int64, numpy=1>, 'Ca': <tf.Tensor: shape=(), dtype=float64, numpy=1.0>, 'Thal': <tf.Tensor: shape=(), dtype=string, numpy=b'reversable'>}
Target: tf.Tensor(True, shape=(), dtype=bool)Let’s batch the datasets:
train_ds = train_ds.batch(32)
val_ds = val_ds.batch(32)
test_ds = test_ds.batch(32)Below, we define 3 utility functions to do the operations:
encode_numerical_feature to apply featurewise normalization to numerical features.
encode_string_categorical_feature to first turn string inputs into integer indices, then one-hot encode these integer indices.
encode_integer_categorical_feature to one-hot encode integer categorical features.
from tensorflow.keras.layers import IntegerLookup
from tensorflow.keras.layers import Normalization
from tensorflow.keras.layers import StringLookup
def encode_numerical_feature(feature, name, dataset):
# Create a Normalization layer for our feature
normalizer = Normalization()
# Prepare a Dataset that only yields our feature
feature_ds = dataset.map(lambda x, y: x[name])
feature_ds = feature_ds.map(lambda x: tf.expand_dims(x, -1))
# Learn the statistics of the data
normalizer.adapt(feature_ds)
# Normalize the input feature
encoded_feature = normalizer(feature)
return encoded_feature
def encode_categorical_feature(feature, name, dataset, is_string):
lookup_class = StringLookup if is_string else IntegerLookup
# Create a lookup layer which will turn strings into integer indices
lookup = lookup_class(output_mode="binary")
# Prepare a Dataset that only yields our feature
feature_ds = dataset.map(lambda x, y: x[name])
feature_ds = feature_ds.map(lambda x: tf.expand_dims(x, -1))
# Learn the set of possible string values and assign them a fixed integer index
lookup.adapt(feature_ds)
# Turn the string input into integer indices
encoded_feature = lookup(feature)
return encoded_feature# Categorical features encoded as integers
Sex = keras.Input(shape = (1,), name = "Sex", dtype = "int64")
Fbs = keras.Input(shape = (1,), name = "Fbs", dtype = "int64")
RestECG = keras.Input(shape = (1,), name = "RestECG", dtype = "int64")
ExAng = keras.Input(shape = (1,), name = "ExAng", dtype="int64")
Ca = keras.Input(shape = (1,), name = "Ca", dtype = "int64")
# Categorical feature encoded as string
ChestPain = keras.Input(shape = (1,), name = "ChestPain", dtype = "string")
Thal = keras.Input(shape = (1,), name = "Thal", dtype = "string")
# Numerical features
Age = keras.Input(shape = (1,), name = "Age")
RestBP = keras.Input(shape = (1,), name = "RestBP")
Chol = keras.Input(shape = (1,), name = "Chol")
MaxHR = keras.Input(shape = (1,), name = "MaxHR")
Oldpeak = keras.Input(shape = (1,), name = "Oldpeak")
Slope = keras.Input(shape = (1,), name = "Slope")
all_inputs = [
Sex,
Fbs,
RestECG,
ExAng,
Ca,
ChestPain,
Thal,
Age,
RestBP,
Chol,
MaxHR,
Oldpeak,
Slope,
]
# Integer categorical features
Sex_encoded = encode_categorical_feature(Sex, "Sex", train_ds, False)
Fbs_encoded = encode_categorical_feature(Fbs, "Fbs", train_ds, False)
RestECG_encoded = encode_categorical_feature(RestECG, "RestECG", train_ds, False)
ExAng_encoded = encode_categorical_feature(ExAng, "ExAng", train_ds, False)
Ca_encoded = encode_categorical_feature(Ca, "Ca", train_ds, False)
# String categorical features
ChestPain_encoded = encode_categorical_feature(ChestPain, "ChestPain", train_ds, True)
Thal_encoded = encode_categorical_feature(Thal, "Thal", train_ds, True)
# Numerical features
Age_encoded = encode_numerical_feature(Age, "Age", train_ds)
RestBP_encoded = encode_numerical_feature(RestBP, "RestBP", train_ds)
Chol_encoded = encode_numerical_feature(Chol, "Chol", train_ds)
MaxHR_encoded = encode_numerical_feature(MaxHR, "MaxHR", train_ds)
Oldpeak_encoded = encode_numerical_feature(Oldpeak, "Oldpeak", train_ds)
Slope_encoded = encode_numerical_feature(Slope, "Slope", train_ds)
all_features = layers.concatenate(
[
Sex_encoded,
Fbs_encoded,
RestECG_encoded,
ExAng_encoded,
Ca_encoded,
ChestPain_encoded,
Thal_encoded,
Age_encoded,
RestBP_encoded,
Chol_encoded,
MaxHR_encoded,
Oldpeak_encoded,
Slope_encoded
]
)import keras_tuner
def build_model(hp):
x = layers.Dense(
# Tune number of units.
units = hp.Int("units", min_value = 16, max_value = 48, step=16),
# Tune the activation function to use.
activation = hp.Choice("activation", ["relu", "tanh"])
)(all_features)
# Tune whether to use dropout
if hp.Boolean("dropout"):
x = layers.Dropout(0.5)(x)
output = layers.Dense(1, activation = "sigmoid")(x)
model = keras.Model(all_inputs, output)
model.compile(
optimizer = keras.optimizers.Adam(
learning_rate = hp.Float("lr", min_value = 1e-4, max_value = 1e-2, sampling="log")
),
loss = "binary_crossentropy",
metrics = ["accuracy"],
)
return model
build_model(keras_tuner.HyperParameters())<keras.engine.functional.Functional object at 0x7fe61fa00760>After defining the search space, we need to select a tuner class to run the search. We may choose from RandomSearch, BayesianOptimization and Hyperband, which correspond to different tuning algorithms. Here we use RandomSearch as an example.
To initialize the tuner, we need to specify several arguments in the initializer.
hypermodel. The model-building function, which is build_model in our case.
objective. The name of the objective to optimize (whether to minimize or maximize is automatically inferred for built-in metrics). We will introduce how to use custom metrics later in this tutorial.
max_trials. The total number of trials to run during the search.
executions_per_trial. The number of models that should be built and fit for each trial. Different trials have different hyperparameter values. The executions within the same trial have the same hyperparameter values. The purpose of having multiple executions per trial is to reduce results variance and therefore be able to more accurately assess the performance of a model. If you want to get results faster, you could set executions_per_trial=1 (single round of training for each model configuration).
overwrite. Control whether to overwrite the previous results in the same directory or resume the previous search instead. Here we set overwrite=True to start a new search and ignore any previous results.
directory. A path to a directory for storing the search results.
project_name. The name of the sub-directory in the directory.
tuner = keras_tuner.RandomSearch(
hypermodel = build_model,
# validation criterion
objective = "val_accuracy",
max_trials = 20,
executions_per_trial = 1,
overwrite = True,
directory = "keras_tuner",
project_name = "heart_mlp",
)We can print a summary of the search space:
tuner.search_space_summary()Search space summary
Default search space size: 4
units (Int)
{'default': None, 'conditions': [], 'min_value': 16, 'max_value': 48, 'step': 16, 'sampling': 'linear'}
activation (Choice)
{'default': 'relu', 'conditions': [], 'values': ['relu', 'tanh'], 'ordered': False}
dropout (Boolean)
{'default': False, 'conditions': []}
lr (Float)
{'default': 0.0001, 'conditions': [], 'min_value': 0.0001, 'max_value': 0.01, 'step': None, 'sampling': 'log'}Start the search
tuner.search(train_ds, epochs = 50, validation_data = val_ds, verbose = 0)Print a summary of the search results.
tuner.results_summary()Results summary
Results in keras_tuner/heart_mlp
Showing 10 best trials
<keras_tuner.engine.objective.Objective object at 0x7fe61ff4ab20>
Trial summary
Hyperparameters:
units: 32
activation: tanh
dropout: False
lr: 0.000229747208769072
Score: 0.8888888955116272
Trial summary
Hyperparameters:
units: 32
activation: tanh
dropout: False
lr: 0.00019831772402689605
Score: 0.8666666746139526
Trial summary
Hyperparameters:
units: 32
activation: relu
dropout: False
lr: 0.0027510201089213765
Score: 0.8444444537162781
Trial summary
Hyperparameters:
units: 32
activation: relu
dropout: False
lr: 0.004283070513582159
Score: 0.8444444537162781
Trial summary
Hyperparameters:
units: 48
activation: relu
dropout: False
lr: 0.0002528056459774514
Score: 0.8444444537162781
Trial summary
Hyperparameters:
units: 16
activation: relu
dropout: False
lr: 0.0012572293292279397
Score: 0.8222222328186035
Trial summary
Hyperparameters:
units: 32
activation: relu
dropout: False
lr: 0.0017036374364839458
Score: 0.8222222328186035
Trial summary
Hyperparameters:
units: 48
activation: tanh
dropout: False
lr: 0.00966955488917958
Score: 0.8222222328186035
Trial summary
Hyperparameters:
units: 48
activation: relu
dropout: False
lr: 0.0011053943066301971
Score: 0.8222222328186035
Trial summary
Hyperparameters:
units: 16
activation: relu
dropout: False
lr: 0.0015929086245058178
Score: 0.8222222328186035Retrieve the best model
# Get the top 2 models.
models = tuner.get_best_models(num_models = 2)
best_model = models[0]
# Build the model.
# Needed for `Sequential` without specified `input_shape`.
best_model.build(input_shape = (None, 13,))
best_model.summary()Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
Sex (InputLayer) [(None, 1)] 0 []
Fbs (InputLayer) [(None, 1)] 0 []
RestECG (InputLayer) [(None, 1)] 0 []
ExAng (InputLayer) [(None, 1)] 0 []
Ca (InputLayer) [(None, 1)] 0 []
ChestPain (InputLayer) [(None, 1)] 0 []
Thal (InputLayer) [(None, 1)] 0 []
Age (InputLayer) [(None, 1)] 0 []
RestBP (InputLayer) [(None, 1)] 0 []
Chol (InputLayer) [(None, 1)] 0 []
MaxHR (InputLayer) [(None, 1)] 0 []
Oldpeak (InputLayer) [(None, 1)] 0 []
Slope (InputLayer) [(None, 1)] 0 []
integer_lookup (IntegerLookup) (None, 3) 0 ['Sex[0][0]']
integer_lookup_1 (IntegerLooku (None, 3) 0 ['Fbs[0][0]']
p)
integer_lookup_2 (IntegerLooku (None, 4) 0 ['RestECG[0][0]']
p)
integer_lookup_3 (IntegerLooku (None, 3) 0 ['ExAng[0][0]']
p)
integer_lookup_4 (IntegerLooku (None, 5) 0 ['Ca[0][0]']
p)
string_lookup (StringLookup) (None, 5) 0 ['ChestPain[0][0]']
string_lookup_1 (StringLookup) (None, 4) 0 ['Thal[0][0]']
normalization (Normalization) (None, 1) 3 ['Age[0][0]']
normalization_1 (Normalization (None, 1) 3 ['RestBP[0][0]']
)
normalization_2 (Normalization (None, 1) 3 ['Chol[0][0]']
)
normalization_3 (Normalization (None, 1) 3 ['MaxHR[0][0]']
)
normalization_4 (Normalization (None, 1) 3 ['Oldpeak[0][0]']
)
normalization_5 (Normalization (None, 1) 3 ['Slope[0][0]']
)
concatenate (Concatenate) (None, 33) 0 ['integer_lookup[0][0]',
'integer_lookup_1[0][0]',
'integer_lookup_2[0][0]',
'integer_lookup_3[0][0]',
'integer_lookup_4[0][0]',
'string_lookup[0][0]',
'string_lookup_1[0][0]',
'normalization[0][0]',
'normalization_1[0][0]',
'normalization_2[0][0]',
'normalization_3[0][0]',
'normalization_4[0][0]',
'normalization_5[0][0]']
dense (Dense) (None, 32) 1088 ['concatenate[0][0]']
dense_1 (Dense) (None, 1) 33 ['dense[0][0]']
==================================================================================================
Total params: 1,139
Trainable params: 1,121
Non-trainable params: 18
__________________________________________________________________________________________________tf.keras.utils.plot_model(best_model, to_file = 'model.png', show_shapes = True)
If we want to train the model with the entire dataset, we may retrieve the best hyperparameters and retrain the model by ourselves.
other_ds = dataframe_to_dataset(Heart_other).batch(32)
best_model.fit(other_ds, epochs = 50, verbose = 2)Epoch 1/50
7/7 - 1s - loss: 0.4523 - accuracy: 0.8288 - 692ms/epoch - 99ms/step
Epoch 2/50
7/7 - 0s - loss: 0.4488 - accuracy: 0.8333 - 12ms/epoch - 2ms/step
Epoch 3/50
7/7 - 0s - loss: 0.4453 - accuracy: 0.8333 - 12ms/epoch - 2ms/step
Epoch 4/50
7/7 - 0s - loss: 0.4424 - accuracy: 0.8333 - 12ms/epoch - 2ms/step
Epoch 5/50
7/7 - 0s - loss: 0.4392 - accuracy: 0.8288 - 12ms/epoch - 2ms/step
Epoch 6/50
7/7 - 0s - loss: 0.4363 - accuracy: 0.8288 - 13ms/epoch - 2ms/step
Epoch 7/50
7/7 - 0s - loss: 0.4333 - accuracy: 0.8243 - 15ms/epoch - 2ms/step
Epoch 8/50
7/7 - 0s - loss: 0.4306 - accuracy: 0.8243 - 14ms/epoch - 2ms/step
Epoch 9/50
7/7 - 0s - loss: 0.4279 - accuracy: 0.8243 - 12ms/epoch - 2ms/step
Epoch 10/50
7/7 - 0s - loss: 0.4252 - accuracy: 0.8288 - 11ms/epoch - 2ms/step
Epoch 11/50
7/7 - 0s - loss: 0.4227 - accuracy: 0.8378 - 11ms/epoch - 2ms/step
Epoch 12/50
7/7 - 0s - loss: 0.4203 - accuracy: 0.8378 - 12ms/epoch - 2ms/step
Epoch 13/50
7/7 - 0s - loss: 0.4179 - accuracy: 0.8378 - 11ms/epoch - 2ms/step
Epoch 14/50
7/7 - 0s - loss: 0.4155 - accuracy: 0.8423 - 11ms/epoch - 2ms/step
Epoch 15/50
7/7 - 0s - loss: 0.4133 - accuracy: 0.8423 - 11ms/epoch - 2ms/step
Epoch 16/50
7/7 - 0s - loss: 0.4110 - accuracy: 0.8423 - 11ms/epoch - 2ms/step
Epoch 17/50
7/7 - 0s - loss: 0.4091 - accuracy: 0.8423 - 11ms/epoch - 2ms/step
Epoch 18/50
7/7 - 0s - loss: 0.4067 - accuracy: 0.8423 - 11ms/epoch - 2ms/step
Epoch 19/50
7/7 - 0s - loss: 0.4049 - accuracy: 0.8468 - 11ms/epoch - 2ms/step
Epoch 20/50
7/7 - 0s - loss: 0.4030 - accuracy: 0.8468 - 10ms/epoch - 1ms/step
Epoch 21/50
7/7 - 0s - loss: 0.4010 - accuracy: 0.8423 - 10ms/epoch - 1ms/step
Epoch 22/50
7/7 - 0s - loss: 0.3992 - accuracy: 0.8423 - 11ms/epoch - 2ms/step
Epoch 23/50
7/7 - 0s - loss: 0.3973 - accuracy: 0.8468 - 11ms/epoch - 2ms/step
Epoch 24/50
7/7 - 0s - loss: 0.3954 - accuracy: 0.8468 - 11ms/epoch - 2ms/step
Epoch 25/50
7/7 - 0s - loss: 0.3939 - accuracy: 0.8514 - 9ms/epoch - 1ms/step
Epoch 26/50
7/7 - 0s - loss: 0.3921 - accuracy: 0.8468 - 10ms/epoch - 1ms/step
Epoch 27/50
7/7 - 0s - loss: 0.3905 - accuracy: 0.8468 - 10ms/epoch - 1ms/step
Epoch 28/50
7/7 - 0s - loss: 0.3888 - accuracy: 0.8468 - 10ms/epoch - 1ms/step
Epoch 29/50
7/7 - 0s - loss: 0.3873 - accuracy: 0.8468 - 10ms/epoch - 1ms/step
Epoch 30/50
7/7 - 0s - loss: 0.3858 - accuracy: 0.8423 - 10ms/epoch - 1ms/step
Epoch 31/50
7/7 - 0s - loss: 0.3842 - accuracy: 0.8423 - 10ms/epoch - 1ms/step
Epoch 32/50
7/7 - 0s - loss: 0.3829 - accuracy: 0.8423 - 10ms/epoch - 1ms/step
Epoch 33/50
7/7 - 0s - loss: 0.3814 - accuracy: 0.8423 - 10ms/epoch - 1ms/step
Epoch 34/50
7/7 - 0s - loss: 0.3801 - accuracy: 0.8378 - 10ms/epoch - 1ms/step
Epoch 35/50
7/7 - 0s - loss: 0.3787 - accuracy: 0.8378 - 10ms/epoch - 1ms/step
Epoch 36/50
7/7 - 0s - loss: 0.3773 - accuracy: 0.8378 - 10ms/epoch - 1ms/step
Epoch 37/50
7/7 - 0s - loss: 0.3762 - accuracy: 0.8378 - 9ms/epoch - 1ms/step
Epoch 38/50
7/7 - 0s - loss: 0.3749 - accuracy: 0.8378 - 10ms/epoch - 1ms/step
Epoch 39/50
7/7 - 0s - loss: 0.3736 - accuracy: 0.8378 - 10ms/epoch - 1ms/step
Epoch 40/50
7/7 - 0s - loss: 0.3723 - accuracy: 0.8378 - 10ms/epoch - 1ms/step
Epoch 41/50
7/7 - 0s - loss: 0.3712 - accuracy: 0.8378 - 10ms/epoch - 1ms/step
Epoch 42/50
7/7 - 0s - loss: 0.3700 - accuracy: 0.8378 - 9ms/epoch - 1ms/step
Epoch 43/50
7/7 - 0s - loss: 0.3689 - accuracy: 0.8378 - 10ms/epoch - 1ms/step
Epoch 44/50
7/7 - 0s - loss: 0.3677 - accuracy: 0.8378 - 11ms/epoch - 2ms/step
Epoch 45/50
7/7 - 0s - loss: 0.3667 - accuracy: 0.8378 - 10ms/epoch - 1ms/step
Epoch 46/50
7/7 - 0s - loss: 0.3656 - accuracy: 0.8378 - 10ms/epoch - 1ms/step
Epoch 47/50
7/7 - 0s - loss: 0.3646 - accuracy: 0.8378 - 10ms/epoch - 1ms/step
Epoch 48/50
7/7 - 0s - loss: 0.3637 - accuracy: 0.8423 - 10ms/epoch - 1ms/step
Epoch 49/50
7/7 - 0s - loss: 0.3627 - accuracy: 0.8378 - 13ms/epoch - 2ms/step
Epoch 50/50
7/7 - 0s - loss: 0.3616 - accuracy: 0.8378 - 11ms/epoch - 2ms/step
<keras.callbacks.History object at 0x7fe6224c4670>score, acc = best_model.evaluate(test_ds, verbose = 2)3/3 - 0s - loss: 0.3286 - accuracy: 0.8800 - 248ms/epoch - 83ms/stepprint('Test score:', score)Test score: 0.32856565713882446print('Test accuracy:', acc)Test accuracy: 0.8799999952316284