Begin Working w/ TPU

This helps you to get the TPU instance, TPU strategy, load the training dataset, validation dataset & test dataset from their TFRecords file & GCS_DS_PATH.

To get all the required utilities, use the following line of code.

Available Methods & Functionalities =>

  1. define_tpu_strategy

  2. get_labeled_tfrecord_format

  3. get_unlabeled_tfrecord_format

  4. get_training_dataset

  5. get_validation_dataset

  6. get_test_dataset

define_tpu_strategy
This returns the tpu instance and the tpu strategy.

get_labeled_tfrecord_format

This sets the Labeled TFRecord Data Format to be read by get_training_dataset or get_validation_dataset

get_unlabeled_tfrecord_format

This sets the Unlabeled TFRecord Data Format to be read by get_test_dataset

 

 

 

 

get_training_dataset
Helps you load the tfrecords file (TRAINING DATASET).

 

Description =>
     GCS_DS_PATH - The GCS Bucket Path of the tfrecords dataset.
     train_tfrec_path - the train tfrecords filename path. eg. '/train.tfrecords'
     BATCH_SIZE - Select the batch size for the images to load in the training dataset instance.

 

Returns =>
A tfrecords dataset instance which can be fed to model training as the training dataset.

get_validation_dataset
Helps you load the tfrecords file (VALIDATION DATASET).

Description =>
     GCS_DS_PATH - The GCS Bucket Path of the tfrecords dataset.
     val_tfrec_path - the validation tfrecords filename path. eg. '/val.tfrecords'
     BATCH_SIZE - Select the batch size for the images to load in the validation dataset instance.

Returns =>
A tfrecords dataset instance which can be fed to model training as the validation dataset.

get_test_dataset
Helps you load the tfrecords file (TEST DATASET).

 

Description =>
     GCS_DS_PATH - The GCS Bucket Path of the tfrecords dataset.
     test_tfrec_path - the test tfrecords filename path. eg. '/test.tfrecords'
     BATCH_SIZE - Select the batch size for the images to load in the test dataset instance.

Returns =>
A tfrecords dataset instance which can be used for prediction as test dataset.

train_dataset = get_training_dataset(GCS_DS_PATH, train_tfrec_path, BATCH_SIZE)

strategy, tpu = define_tpu_strategy(mixed_precision = False, xla_accelerate = False)

from quick_ml.begin_tpu import define_tpu_strategy, get_training_dataset, get_validation_dataset, get_test_dataset

val_dataset = get_validation_dataset(GCS_DS_PATH, val_tfrec_path, BATCH_SIZE)

test_dataset = get_test_dataset(GCS_DS_PATH, test_tfrec_path, BATCH_SIZE)

# please name the variables dictionary_labeled and IMAGE_SIZE as it is & mind the format of their value allocation

dictionary_labeled = "{'image' : tf.io.FixedLenFeature([], tf.string), 'label' : tf.io.FixedLenFeature([], tf.int64)}"
IMAGE_SIZE = "192,192"

from quick_ml.begin_tpu import get_labeled_tfrecord_format
get_labeled_tfrecord_format(dictionary_labeled, IMAGE_SIZE)

# please name the variables dictionary_unlabeled and IMAGE_SIZE as it is & mind the format of their value allocation

dictionary_unlabeled = "{'image' : tf.io.FixedLenFeature([], tf.string), 'idnum' : tf.io.FixedLenFeature([], tf.int64)}"
IMAGE_SIZE = "192,192"

from quick_ml.begin_tpu import get_unlabeled_tfrecord_format
get_unlabeled_tfrecord_format(dictionary_unlabeled, IMAGE_SIZE)

 

Was this helpful?