Use TensorFlow to predict handwritten digits | ![]() |
---|---|
![]() |
This notebook contains steps and code to demonstrate Deep Learning model training and scoring in the Watson Machine Learning service. This notebook introduces commands for getting data, training_definition persistance to Watson Machine Learning repository, model training, deployment and scoring.
Some familiarity with Python is helpful. This notebook uses Python 3 and Watson Studio environments.
In this notebook, you will learn how to:
Before you use the sample code in this notebook, you must perform the following setup tasks:
Create a Watson Machine Learning (WML) Service instance (a free plan is offered and information about how to create the instance is here).
Create a Cloud Object Storage (COS) instance (a lite plan is offered and information about how to order storage is here).
Note: When using Watson Studio, you already have a COS instance associated with the project you are running the notebook in.
Create new credentials with HMAC:
This configuration parameter adds the following section to the instance credentials, (for use later in this notebook):
"cos_hmac_keys": {
"access_key_id": "-------",
"secret_access_key": "-------"
}
Install the boto library. This library allows Python developers to manage Cloud Object Storage (COS).
Tip: If ibm_boto3
is not preinstalled in you environment, run the following command to install it:
# Run the command if ibm_boto3 is not installed.
!pip install ibm-cos-sdk
Requirement not upgraded as not directly required: ibm-cos-sdk in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages Requirement not upgraded as not directly required: ibm-cos-sdk-s3transfer==2.*,>=2.0.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from ibm-cos-sdk) Requirement not upgraded as not directly required: ibm-cos-sdk-core==2.*,>=2.0.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from ibm-cos-sdk) Requirement not upgraded as not directly required: docutils>=0.10 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from ibm-cos-sdk-core==2.*,>=2.0.0->ibm-cos-sdk) Requirement not upgraded as not directly required: python-dateutil<3.0.0,>=2.1 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from ibm-cos-sdk-core==2.*,>=2.0.0->ibm-cos-sdk) Requirement not upgraded as not directly required: jmespath<1.0.0,>=0.7.1 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from ibm-cos-sdk-core==2.*,>=2.0.0->ibm-cos-sdk) Requirement not upgraded as not directly required: six>=1.5 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from python-dateutil<3.0.0,>=2.1->ibm-cos-sdk-core==2.*,>=2.0.0->ibm-cos-sdk)
# Install the boto library.
import ibm_boto3
from ibm_botocore.client import Config
Enter your COS credentials in the following cell.
You can find these credentials in your COS instance dashboard under the Service credentials tab.
Note: the HMAC key, described in set up the environment is included in these credentials.
cos_credentials = {
"apikey": "***",
"cos_hmac_keys": {
"access_key_id": "***",
"secret_access_key": "***"
},
"endpoints": "https://cos-service.bluemix.net/endpoints",
"iam_apikey_description": "Auto generated apikey during resource-key operation for Instance - crn:v1:bluemix:public:cloud-object-storage:global:a/07a95aa44e6124e8b320b70cf88033fa:876e5285-4bef-4cf3-a89b-595e19648c7c::",
"iam_apikey_name": "auto-generated-apikey-19a79dae-6a58-4b4f-878f-6839b711523f",
"iam_role_crn": "crn:v1:bluemix:public:iam::::serviceRole:Writer",
"iam_serviceid_crn": "crn:v1:bluemix:public:iam-identity::a/07a95aa44e6124e8b320b70cf88033fa::serviceid:ServiceId-3f2cccee-61ec-4147-8732-9f58479ba26a",
"resource_instance_id": "crn:v1:bluemix:public:cloud-object-storage:global:a/07a95aa44e6124e8b320b70cf88033fa:876e5285-4bef-4cf3-a89b-595e19648c7c::"
}
# The code was removed by Watson Studio for sharing.
Define the endpoint.
To do this, go to the Endpoint tab in the COS instance's dashboard to get the endpoint information, then enter it in the cell below:
# Define endpoint information.
service_endpoint = 'https://s3-api.us-geo.objectstorage.softlayer.net'
You also need the IBM Cloud authorization endpoint to be able to create COS resource object.
# Define the authorization endpoint.
auth_endpoint = 'https://iam.bluemix.net/oidc/token'
Create a Boto resource to be able to write data to COS.
# Create a COS resource.
cos = ibm_boto3.resource('s3',
ibm_api_key_id=cos_credentials['apikey'],
ibm_service_instance_id=cos_credentials['resource_instance_id'],
ibm_auth_endpoint=auth_endpoint,
config=Config(signature_version='oauth'),
endpoint_url=service_endpoint)
Create two buckets, which you will use to store training data and training results.
Note: The bucket names must be unique.
from uuid import uuid4
bucket_uid = str(uuid4())
buckets = ['training-data-' + bucket_uid, 'training-results-' + bucket_uid]
for bucket in buckets:
if not cos.Bucket(bucket) in cos.buckets.all():
print('Creating bucket "{}"...'.format(bucket))
try:
cos.create_bucket(Bucket=bucket)
except ibm_boto3.exceptions.ibm_botocore.client.ClientError as e:
print('Error: {}.'.format(e.response['Error']['Message']))
Creating bucket "training-data-eacf90dc-f69a-437c-8f43-5d36aa59abf4"... Creating bucket "training-results-eacf90dc-f69a-437c-8f43-5d36aa59abf4"...
Now you should have 2 buckets.
# Display a list of created buckets.
print(list(cos.buckets.all()))
[s3.Bucket(name='9fff2508-60c5-4e75-835b-210aa00ff010-style-data'), s3.Bucket(name='9fff2508-60c5-4e75-835b-210aa00ff010-style-results'), s3.Bucket(name='aiopenscalepythonclient-donotdelete-pr-hp7rm9ptqfgzw7'), s3.Bucket(name='aispheredemo-donotdelete-pr-s7lhzejm05qqne'), s3.Bucket(name='aispherescenario-datacatalog-sydghgvlq'), s3.Bucket(name='ale-kupa-in'), s3.Bucket(name='cars4you-train-data'), s3.Bucket(name='cifar-keras-nnm-input'), s3.Bucket(name='coreml-donotdelete-pr-5lprxel1c6jrfp'), s3.Bucket(name='coreml-mnist-input-data'), s3.Bucket(name='creditcardfraud545b518d4ec34681aa7e8e8680b486d5'), s3.Bucket(name='datasphereworkshops-donotdelete-pr-xfdm0p7mjo5ms6'), s3.Bucket(name='mnist-keras-data-example'), s3.Bucket(name='mnist-keras-data-example2018-06-01-19-36'), s3.Bucket(name='mnist-keras-results-example'), s3.Bucket(name='mnist-keras-results-example2018-06-01-19-36'), s3.Bucket(name='style-data-example'), s3.Bucket(name='style-data-example-workshops'), s3.Bucket(name='style-results-example'), s3.Bucket(name='style-results-example-workshops'), s3.Bucket(name='train-data-experiment-2018-03-28-09-34'), s3.Bucket(name='train-data-experiment-2018-05-30-14-26'), s3.Bucket(name='train-data-experiment-2018-08-16-13-11'), s3.Bucket(name='train-result-experiment-2018-03-28-09-34'), s3.Bucket(name='train-result-experiment-2018-05-30-14-26'), s3.Bucket(name='train-result-experiment-2018-08-16-13-11'), s3.Bucket(name='training-data-38a61fd8-a250-4a82-9277-ef51f88bfadb'), s3.Bucket(name='training-data-47687ebc-1597-4cdb-b692-51600c0fa77e'), s3.Bucket(name='training-data-eacf90dc-f69a-437c-8f43-5d36aa59abf4'), s3.Bucket(name='training-results-38a61fd8-a250-4a82-9277-ef51f88bfadb'), s3.Bucket(name='training-results-47687ebc-1597-4cdb-b692-51600c0fa77e'), s3.Bucket(name='training-results-eacf90dc-f69a-437c-8f43-5d36aa59abf4'), s3.Bucket(name='watsonstudiosamplenotebooks2ec122d7327248859e8d39c03f931082'), s3.Bucket(name='xxx-donotdelete-pr-ly02ysmxmvyqcr')]
Download the training data and upload it to the 'training-data' bucket. First, create a list of links for the training data set.
# Create a list of links.
data_links = ['http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz',
'http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz',
'http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz',
'http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz']
The code in the next cell uploads files from links to your COS.
# Upload files from the links to COS
from urllib.request import urlopen
bucket_obj = cos.Bucket(buckets[0])
for data_link in data_links:
filename=data_link.split('/')[-1]
print('Uploading data {}...'.format(filename))
with urlopen(data_link) as data:
bucket_obj.upload_fileobj(data, filename)
print('{} is uploaded.'.format(filename))
Uploading data train-images-idx3-ubyte.gz... train-images-idx3-ubyte.gz is uploaded. Uploading data train-labels-idx1-ubyte.gz... train-labels-idx1-ubyte.gz is uploaded. Uploading data t10k-images-idx3-ubyte.gz... t10k-images-idx3-ubyte.gz is uploaded. Uploading data t10k-labels-idx1-ubyte.gz... t10k-labels-idx1-ubyte.gz is uploaded.
Have a look at the list of the created buckets and their contents.
for bucket_name in buckets:
print(bucket_name)
bucket_obj = cos.Bucket(bucket_name)
for obj in bucket_obj.objects.all():
print(" File: {}, {:4.2f}kB".format(obj.key, obj.size/1024))
training-data-eacf90dc-f69a-437c-8f43-5d36aa59abf4 File: t10k-images-idx3-ubyte.gz, 1610.23kB File: t10k-labels-idx1-ubyte.gz, 4.44kB File: train-images-idx3-ubyte.gz, 9680.10kB File: train-labels-idx1-ubyte.gz, 28.20kB training-results-eacf90dc-f69a-437c-8f43-5d36aa59abf4
You are done with COS, and you are ready to train your model!
Import the libraries you need to work with your WML instance.
Hint: You may also need to install wget
using the following command !pip install wget
import urllib3, requests, json, base64, time, os, wget
Authenticate to the Watson Machine Learning (WML) service on IBM Cloud.
Tip: Authentication information (your credentials) can be found in the Service credentials tab of the service instance that you created on IBM Cloud. If there are no credentials listed for your instance in Service credentials, click New credential (+) and enter the information required to generate new authentication information.
Action: Enter your WML service instance credentials here.
wml_credentials = {
"url": "https://ibm-watson-ml.mybluemix.net",
"access_key": "vgXPA8k5Ld613WJP/S0ijQZtiitjuwsj9qpVt+nvl9DFl8PK/67XgmnreKabd2ZPpxSFKe9cZoFYLlzgPf++qpWZYcc+6fawL9S0V+2V79Adc+zik+ZHJYrsBRl9GAcs",
"username": "***",
"password": "***",
"instance_id": "***"
}
# The code was removed by Watson Studio for sharing.
watson-machine-learning-client
library from pypi.¶!rm -rf $PIP_BUILD/watson-machine-learning-client
!pip install watson-machine-learning-client --upgrade
Requirement already up-to-date: watson-machine-learning-client in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages Requirement not upgraded as not directly required: requests in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client) Requirement not upgraded as not directly required: tqdm in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client) Requirement not upgraded as not directly required: ibm-cos-sdk in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client) Requirement not upgraded as not directly required: lomond in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client) Requirement not upgraded as not directly required: urllib3 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client) Requirement not upgraded as not directly required: tabulate in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client) Requirement not upgraded as not directly required: pandas in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client) Requirement not upgraded as not directly required: certifi in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client) Requirement not upgraded as not directly required: chardet<3.1.0,>=3.0.2 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from requests->watson-machine-learning-client) Requirement not upgraded as not directly required: idna<2.7,>=2.5 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from requests->watson-machine-learning-client) Requirement not upgraded as not directly required: ibm-cos-sdk-core==2.*,>=2.0.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from ibm-cos-sdk->watson-machine-learning-client) Requirement not upgraded as not directly required: ibm-cos-sdk-s3transfer==2.*,>=2.0.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from ibm-cos-sdk->watson-machine-learning-client) Requirement not upgraded as not directly required: six>=1.10.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from lomond->watson-machine-learning-client) Requirement not upgraded as not directly required: python-dateutil>=2 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client) Requirement not upgraded as not directly required: pytz>=2011k in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client) Requirement not upgraded as not directly required: numpy>=1.9.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client) Requirement not upgraded as not directly required: jmespath<1.0.0,>=0.7.1 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from ibm-cos-sdk-core==2.*,>=2.0.0->ibm-cos-sdk->watson-machine-learning-client) Requirement not upgraded as not directly required: docutils>=0.10 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from ibm-cos-sdk-core==2.*,>=2.0.0->ibm-cos-sdk->watson-machine-learning-client)
watson-machine-learning-client
and authenticate to the service instance.¶from watson_machine_learning_client import WatsonMachineLearningAPIClient
2018-08-20 13:06:56,609 - watson_machine_learning_client.metanames - WARNING - 'AUTHOR_EMAIL' meta prop is deprecated. It will be ignored. 2018-08-20 13:06:59,394 - watson_machine_learning_client.metanames - WARNING - 'AUTHOR_EMAIL' meta prop is deprecated. It will be ignored. 2018-08-20 13:10:08,158 - watson_machine_learning_client.metanames - WARNING - 'AUTHOR_EMAIL' meta prop is deprecated. It will be ignored.
Note: A deprecation warning is returned from scikit-learn package that does not impact watson machine learning client functionalities.
client = WatsonMachineLearningAPIClient(wml_credentials)
# Display the client version number.
print(client.version)
1.0.302
Note: watson-machine-learning-client
documentation can be found here.
In this section you:
model_definition_metadata = {
client.repository.DefinitionMetaNames.NAME: "My definition name",
client.repository.DefinitionMetaNames.DESCRIPTION: "My description",
client.repository.DefinitionMetaNames.AUTHOR_NAME: "John Smith",
client.repository.DefinitionMetaNames.FRAMEWORK_NAME: "tensorflow",
client.repository.DefinitionMetaNames.FRAMEWORK_VERSION: "1.5",
client.repository.DefinitionMetaNames.RUNTIME_NAME: "python",
client.repository.DefinitionMetaNames.RUNTIME_VERSION: "3.5",
client.repository.DefinitionMetaNames.EXECUTION_COMMAND: "python3 tensorflow_mnist_softmax.py --trainingIters 20"
}
filename='tf-softmax-model.zip'
if not os.path.isfile(filename):
filename = wget.download('https://github.com/pmservice/wml-sample-models/raw/master/tensorflow/hand-written-digit-recognition/definition/tf-softmax-model.zip')
print(filename)
tf-softmax-model.zip
definition_details = client.repository.store_definition(filename, model_definition_metadata)
definition_uid = client.repository.get_definition_uid(definition_details)
# Display the training definition uid.
print(definition_uid)
65481c9c-840b-46d7-8dfa-4376ef9c7f4a
In this section, learn how to:
TRAINING_DATA_REFERENCE
- references the uploaded training data.TRAINING_RESULTS_REFERENCE
- location where trained model will be saved.Note Your COS credentials are referenced in this code.
# Configure the training metadata for the TRAINING_DATA_REFERENCE and TRAINING_RESULTS_REFERENCE.
training_configuration_metadata = {
client.training.ConfigurationMetaNames.NAME: "Hand-written Digit Recognition",
client.training.ConfigurationMetaNames.AUTHOR_NAME: "John Smith",
client.training.ConfigurationMetaNames.DESCRIPTION: "Hand-written Digit Recognition training",
client.training.ConfigurationMetaNames.COMPUTE_CONFIGURATION: {"name": "k80"},
client.training.ConfigurationMetaNames.TRAINING_DATA_REFERENCE: {
"connection": {
"endpoint_url": service_endpoint,
"access_key_id": cos_credentials['cos_hmac_keys']['access_key_id'],
"secret_access_key": cos_credentials['cos_hmac_keys']['secret_access_key']
},
"source": {
"bucket": buckets[0],
},
"type": "s3"
},
client.training.ConfigurationMetaNames.TRAINING_RESULTS_REFERENCE: {
"connection": {
"endpoint_url": service_endpoint,
"access_key_id": cos_credentials['cos_hmac_keys']['access_key_id'],
"secret_access_key": cos_credentials['cos_hmac_keys']['secret_access_key']
},
"target": {
"bucket": buckets[1],
},
"type": "s3"
},
}
To run the training in the background, set the optional parameter asynchronous=True
(or remove it). In this case the parameter has been removed.
Note: To run the training in active mode, set asynchronous=False
.
training_run_details = client.training.run(definition_uid, training_configuration_metadata)
training_run_guid_async = client.training.get_run_uid(training_run_details)
Check the status of the training run by calling the method the next cell:
# Get training run status.
status = client.training.get_status(training_run_guid_async)
print(json.dumps(status, indent=2))
{ "submitted_at": "2018-08-20T13:07:03Z", "state": "pending", "current_at": "2018-08-20T13:07:04Z", "metrics": [] }
Run the cell below to monitor the training log.
client.training.monitor_logs(training_run_guid_async)
######################################################## Log monitor started for training run: training-C00cT2tiR ######################################################## Training with training/test data at: DATA_DIR: /mnt/data/training-data-eacf90dc-f69a-437c-8f43-5d36aa59abf4 MODEL_DIR: /job/model-code TRAINING_JOB: TRAINING_COMMAND: python3 tensorflow_mnist_softmax.py --trainingIters 20 Storing trained model at: RESULT_DIR: /mnt/results/training-results-eacf90dc-f69a-437c-8f43-5d36aa59abf4/training-C00cT2tiR Mon Aug 20 13:08:12 UTC 2018: Running Tensorflow job /usr/local/lib/python3.5/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`. from ._conv import register_converters as _register_converters WARNING:tensorflow:From tensorflow_mnist_softmax.py:66: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version. Instructions for updating: Future major versions of TensorFlow will allow gradients to flow into the labels input on backprop by default. See tf.nn.softmax_cross_entropy_with_logits_v2. 2018-08-20 13:08:16.928487: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA 2018-08-20 13:08:17.106586: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1105] Found device 0 with properties: name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235 pciBusID: 0000:83:00.0 totalMemory: 11.17GiB freeMemory: 11.10GiB 2018-08-20 13:08:17.106632: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1195] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: Tesla K80, pci bus id: 0000:83:00.0, compute capability: 3.7) Start model training Optimization Finished! Testing Accuracy: 0.9168 classification_signature content: inputs { key: "inputs" value { name: "Placeholder:0" dtype: DT_FLOAT tensor_shape { dim { size: -1 } dim { size: 784 } } } } outputs { key: "classes" value { name: "predictor:0" dtype: DT_INT64 tensor_shape { dim { size: -1 } } } } method_name: "tensorflow/serving/classify" ----------------- Log monitor done. -----------------
You can cancel the training run by calling the method below.
# Save model to WML repository.
saved_model_details = client.repository.store_model(training_run_guid_async, {"name": "My cool mnist model"})
# Display location of saved model.
print("Url: " + client.repository.get_model_url(saved_model_details))
Url: https://us-south.ml.cloud.ibm.com/v3/wml_instances/3f6e5c2b-4880-46aa-9d79-62e90ccc9d56/published_models/9039c349-b6e5-49ce-9212-bab6d5a7a72e
Tip: Run the code below to extract the stored model uid from the dictionary.
# Get and display model uid.
model_uid = client.repository.get_model_uid(saved_model_details)
print("Saved model uid: " + model_uid)
Saved model uid: 9039c349-b6e5-49ce-9212-bab6d5a7a72e
client.repository.list_models()
------------------------------------ ------------------------------------ ------------------------ ----------------- GUID NAME CREATED FRAMEWORK 9039c349-b6e5-49ce-9212-bab6d5a7a72e My cool mnist model 2018-08-20T13:10:08.469Z tensorflow-1.5 45180921-025b-4881-9ef0-2cff85c716d6 MNIST best model 2018-08-20T12:09:12.954Z tensorflow-1.5 9982062e-f598-4cc6-b6a6-fb11c0427aba MNIST best model 2018-08-08T17:13:28.977Z tensorflow-1.5 19e95da9-520f-49bc-9e0c-227a5e74e130 Advisor productivity model 2018-06-13T18:02:54.960Z mllib-2.1 cd7602c8-6630-4c89-8438-2b41314c1af8 Advisor productivity model 2018-06-13T17:19:08.751Z mllib-2.1 3dddfb49-51de-471e-b9df-cf7174630845 Advisor productivity model 2018-06-13T16:07:48.140Z mllib-2.1 84a49f46-8e4e-41e4-958b-7e3073820d62 Advisor productivity model 2018-06-13T04:13:06.731Z mllib-2.1 fe00574f-95a2-4dd3-b29f-30d071734662 Advisor productivity model 2018-06-12T21:07:23.922Z mllib-2.1 82d124ab-f174-424b-bf93-8b4240df2875 Advisor productivity model 2018-06-12T18:11:49.350Z mllib-2.1 0498a7d4-56fd-43bf-92ef-1bebd6b057ef Advisor productivity model 2018-06-12T16:30:22.586Z mllib-2.1 1ab43c12-4313-46e9-8a1c-09bdb8004af4 Product line model 2018-06-12T07:56:20.689Z mllib-2.1 414fd52b-74a3-4d5e-ab5f-1288a0df83db Product line model 2018-06-11T08:19:51.753Z mllib-2.1 eec29270-e1f1-416a-9dce-9cf794f083c3 Product line model 2018-06-09T11:34:21.889Z mllib-2.1 3dffd8f3-e1e3-43f9-9497-fe2d5714ff8d Product line model 2018-06-05T17:27:29.780Z mllib-2.1 2ff256ec-1e40-45a5-8aba-7d83f2087ae7 Product line model 2018-05-31T20:09:26.907Z mllib-2.1 6ad5ae64-25ba-402d-8f54-384b4f9b9d08 Product line model 2018-05-30T11:21:37.085Z mllib-2.1 23f06316-1c4e-492a-9760-e0f8ccdbac33 Product line model 2018-05-30T06:13:21.145Z mllib-2.1 2c065114-e8ca-4f5a-819f-b05ca8dbe1f0 Product line model 2018-05-29T11:52:52.012Z mllib-2.1 6e589f3c-4b70-4816-a47a-4a345cc81024 Product line model 2018-05-29T11:29:25.962Z mllib-2.1 e963442c-5445-4fd8-bcaa-a7f8590f1eb3 Product line model 2018-05-24T15:46:27.617Z mllib-2.1 e6bc671d-7731-4149-ae28-62be0d09ac93 MNIST best model 2018-05-24T12:59:05.837Z tensorflow-1.5 267e805c-5639-435b-b4de-8c7be5e12700 Core ML - keras mnist model 2018-04-24T14:36:39.346Z tensorflow-1.5 b7bcb71e-302b-4087-bcde-3be6e4188e03 Core ML - keras mnist model 2018-04-23T13:55:25.119Z tensorflow-1.5 2e1434e3-21dd-4f38-b227-cf113ea9d6c7 Core ML - keras mnist model 2018-04-23T13:42:59.695Z tensorflow-1.5 ec476762-d41b-4483-8e92-c13f3df9b356 Core ML - keras mnist model 2018-04-23T13:36:47.517Z tensorflow-1.5 49f64c52-3cc3-47d9-bca5-ec9aea294c74 Core ML - keras mnist model 2018-04-23T13:33:14.643Z tensorflow-1.5 b790fb08-a7a3-4c4b-9639-494e8038d924 Credit Card Fraud Detection Scikit 2018-04-20T09:21:56.225Z scikit-learn-0.19 6e7aaadd-b86f-4751-b0ee-a74f38938b10 Fraud Detection - Credit Card (SPSS) 2018-04-09T18:40:01.883Z spss-modeler-18.1 51aa4ef2-81e7-44bb-854d-1e459ac9f7ca Credit Card Fraud Detection 2018-04-06T12:17:29.820Z scikit-learn-0.19 9faf9502-87ca-4a15-8581-060cc84ccc5b Sentiment Prediction Model 2018-03-29T10:37:16.069Z mllib-2.1 70a8b8ad-ef04-4d9f-a1e7-679719577b62 Product line model 2018-03-29T09:21:46.826Z mllib-2.1 298832f7-d21d-4a07-a028-b5716dba1264 Best Heart Drug Selection 2018-03-29T08:35:31.303Z mllib-2.1 0da45aa2-5378-4dc1-8617-3d209499f466 my spark model 2018-03-28T14:49:01.055Z mllib-2.1 9fb96bc1-bc12-4717-a03f-9b009a32d578 SPSS model for Churn prediction 2018-03-28T13:20:40.791Z spss-modeler-18.1 5bcf67ab-cd88-49c9-99fd-99dde60eff70 SPSS model for Churn prediction 2018-03-28T13:11:28.164Z spss-modeler-17.1 91149430-5588-4ee6-ac6a-695cd2392aa8 My cool mnist model tf-ddl 2018-03-28T11:15:59.813Z tensorflow-1.5 b6e59ce5-7d24-4ce1-90f8-f6a918ad71ca My cool mnist model tf-distributed 2018-03-28T11:15:56.061Z tensorflow-1.5 84070d90-6db7-4a12-8da6-26aaf18e9378 My cool mnist model tf-horovod 2018-03-28T11:15:53.446Z tensorflow-1.5 97822727-4bd1-4df0-95df-8ff9536ea01d My cool mnist model 2018-03-28T09:02:31.509Z tensorflow-1.5 a0cb79ca-e937-4cbf-b7e5-23dad2d2ecec drug-selection 2018-03-16T18:10:25.046Z mllib-2.0 327242a2-d2c0-400f-b673-4f33ad7ec17d product-line-prediction 2018-03-16T18:08:31.260Z mllib-2.0 f7219c8f-fccb-4d26-a1bd-f51544090eae training-S3ttnlgig_2 best model 2018-03-16T13:32:33.747Z tensorflow-1.5 861b6b73-6861-49c0-867c-d171d6d9d545 MNIST best model 2018-03-16T13:00:40.461Z tensorflow-1.5 c755c740-dab6-4d5d-b873-1060885d55d5 MNIST best model 2018-03-16T10:12:31.990Z tensorflow-1.5 a5a952e1-fa6f-4c64-88f7-293b367b7e1f MNIST best model 2018-03-16T10:12:16.773Z tensorflow-1.5 43986a55-0f0e-493b-9b98-0712f918b0ba MNIST CNN model 2018-03-15T16:12:06.503Z tensorflow-1.5 ------------------------------------ ------------------------------------ ------------------------ -----------------
Run the code in the cell below to delete train runs and results:
Run the code in the following cell to deploy the stored model as a web service (online):
# Create online deployment.
deployment_details = client.deployments.create(model_uid, "Mnist model deployment")
####################################################################################### Synchronous deployment creation for uid: '9039c349-b6e5-49ce-9212-bab6d5a7a72e' started ####################################################################################### INITIALIZING DEPLOY_SUCCESS ------------------------------------------------------------------------------------------------ Successfully finished deployment creation, deployment_uid='0befc8fd-3f16-4e8d-9f14-3b890be4b4c8' ------------------------------------------------------------------------------------------------
Now, extract the url endpoint, scoring_url, which will be used to send scoring requests.
# Extract endpoint url and display it.
scoring_url = client.deployments.get_scoring_url(deployment_details)
print(scoring_url)
https://ibm-watson-ml.mybluemix.net/v3/wml_instances/3f6e5c2b-4880-46aa-9d79-62e90ccc9d56/deployments/0befc8fd-3f16-4e8d-9f14-3b890be4b4c8/online
Prepare sample scoring data to score deployed model.
image1 = [0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.639,0.757,0.596,0.361,0.200,0.200,0.200,0.200,0.122,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.082,0.875,0.992,0.988,0.992,0.988,0.992,0.988,0.992,0.753,0.322,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.200,0.992,0.400,0.000,0.082,0.400,0.243,0.400,0.400,0.239,0.718,0.157,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.200,0.988,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.239,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.200,0.992,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.361,0.988,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.678,0.992,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.992,0.988,0.875,0.796,0.796,0.796,0.322,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.482,0.996,0.992,0.878,0.796,0.796,0.875,0.996,0.278,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.161,0.953,0.992,0.510,0.078,0.000,0.000,0.078,0.992,0.910,0.161,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.596,0.992,0.718,0.000,0.000,0.000,0.000,0.000,0.518,0.992,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.200,0.592,0.078,0.000,0.000,0.000,0.000,0.000,0.200,0.988,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.082,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.443,0.992,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.322,0.718,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.757,0.988,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.796,0.718,0.000,0.000,0.000,0.000,0.000,0.000,0.082,0.996,0.992,0.161,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.082,0.875,0.400,0.000,0.000,0.000,0.000,0.000,0.082,0.796,0.992,0.510,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.800,0.482,0.000,0.000,0.000,0.000,0.161,0.678,0.992,0.796,0.078,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.635,0.875,0.400,0.082,0.400,0.639,0.953,0.992,0.671,0.078,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.161,0.918,0.992,1.000,0.992,1.000,0.675,0.322,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.118,0.514,0.753,0.435,0.196,0.039,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000]
image2 = [0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.196,0.878,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.275,0.114,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.475,0.906,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.580,0.659,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.016,0.765,0.906,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.376,0.824,0.043,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.271,0.988,0.525,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.447,0.988,0.082,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.176,0.925,0.851,0.047,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.753,0.988,0.082,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.659,0.969,0.208,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.071,1.000,0.992,0.082,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.329,0.949,0.827,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.553,0.992,0.741,0.020,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.663,0.988,0.416,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.125,0.910,0.980,0.259,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.059,0.882,0.988,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.525,0.988,0.827,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.086,0.988,0.643,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.663,0.988,0.655,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.035,0.800,0.820,0.071,0.000,0.000,0.000,0.000,0.000,0.000,0.086,0.992,0.992,0.420,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.663,0.988,0.780,0.333,0.333,0.333,0.333,0.506,0.643,0.765,0.988,0.988,0.416,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.161,0.667,0.961,0.988,0.988,0.988,0.988,0.910,0.906,0.984,0.988,0.988,0.035,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.192,0.329,0.329,0.329,0.329,0.000,0.000,0.631,0.988,0.988,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.498,0.988,0.988,0.176,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.502,0.992,0.992,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.498,0.988,0.988,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.529,0.988,0.957,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.910,0.925,0.435,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.702,0.259,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000]
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
for i, image in enumerate([image1, image2]):
plt.subplot(2, 2, i + 1)
plt.axis('off')
plt.imshow( (np.reshape(image, (28, 28)) * 255).astype(np.uint8), cmap=plt.cm.gray_r, interpolation='nearest')
Build a scoring dictionary consisting of two digits and send it to the deployed model to get predictions.
# Build scoring dictionary which has two digits (image1 and image29.
scoring_data = {'values': [image1, image2]}
# send scoring dictionary to deployed model to get predictions
predictions = client.deployments.score(scoring_url, scoring_data)
print("Scoring result: " + str(predictions))
Scoring result: {'values': [5, 4]}
You successfully completed this notebook!
You learned how to use watson-machine-learning-client
to train and score TensorFlow models.
Check out our Online Documentation for a tutorial and more samples, documentation, how-tos, and blog posts.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. "Gradient-based learning applied to document recognition." Proceedings of the IEEE, 86(11):2278-2324, November 1998.
Lukasz Cmielowski, PhD, is a Automation Architect and Data Scientist at IBM with a track record of developing enterprise-level applications that substantially increase the clients' ability to turn data into actionable knowledge.
Copyright © 2017, 2018 IBM. This notebook and its source code are released under the terms of the MIT License.