Models¶

All tower APIs are accesible directly from a tflon.Model instance.

Model API¶

class tflon.model.model.Model(**kwargs)¶

Model base class. User models are implemented by extending this class and overriding the _model method:

class MyModel(Model):

def _model(self, *args, **kwargs):: pass

checkpoint(filename)¶

Generate a tensorflow checkpoint file.

Parameters:	filename (str) – Path to write the checkpoint file

evaluate(source, query=None)¶

Evaluate model metrics on data feed or batch

Keyword Arguments:
Parameters:	source – Either (1) a batch dictionary of name, tensor pairs or (2) a generator, which emits batches (dictionaries of name, tensor pairs)
	query (list or str) – A key or multiple keys specifying which metrics to gather, if None, gets all metrics (default=None)
Returns:	A dictionary of metric names and values
Return type:	dict

featurize(source, batch_size=None)¶

Given a table feed, generate the feature tables required by each module and add them to the table feed

Keyword Arguments:
Parameters:	source (TableFeed) – The input feed
	batch_size (int) – Featurize the data in batches of a given size, if None, featurize all the data at once

feed(batch)¶: Generate a feed_dict from a batch dictionary for evaluating session.run calls

fetch(patterns)¶

Get variable values by regex patterns.

Parameters:	patterns (list) – list of string queries for variable names
Returns:	Variable values as name:value pairs
Return type:	dict

fit(source, trainer, restarts=1, reset=True, source_tables=None, **kwargs)¶

Fit a model to data using a chosen trainer.

This function manages random restarts and delegates training to either _fit_global or _fit_stochastic, which are selected based the type of source.

Parameters:

source – Either a dictionary-like of name:feedable pairs, or an iterator which returns such dictionaries
trainer – A tflon.train.Trainer

Keyword Arguments:

restarts (int) – The number of times to train from a random initialization (default = 1)
reset (bool) – Whether to reset trainable variable initializations before each optimization (default = True)
source_tables – An optional TableFeed object or dictionary of name:Table pairs which is used to initialize model parameters (default = None)
**kwargs – Additional arguments passed to either _fit_global or _fit_stochastic

infer(source, query=None)¶

Perform inference from a data batch or from a data generator.

Keyword Arguments:
Parameters:	source – Either (1) a batch dictionary of name, tensor pairs or (2) a generator, which emits batches (dictionaries of name, tensor pairs)
	query (list or str) – A key or multiple keys specifying which outputs to gather, if None, gets all outputs (default=None)
Returns:	If source is a batch dictionary, return a dictionary of name, tensor pairs for model output. Otherwise, if source is a batch generator, return a generator which emits dictionaries of name, tensor pairs, one output for each input batch
Return type:	dict or iterator

initialize(restore=True)¶

Reset the model variables to their initialized states.

This operation will restore variables to saved states loaded from disk if available.

Parameters:	restore (boolean) – If False, do not use saved variable states even when available (default=True)

classmethod load(filename, **xargs)¶

Load a tflon model pickle file previously saved with Model.save from disk

Parameters:	filename – The pickle file containing the saved tflon model xargs – Extra keyword arguments passed to Model.__init__ this argument can be used to override parameter settings

reset()¶: Reinitialize model trainable variables.

restore(filename)¶

Restore model weights from a tensorflow checkpoint file.

Parameters:	filename (str) – A path to a checkpoint file

save(filename)¶

Save a tflon model pickle file. The resulting file contains a tuple data structure with the following three variables:

(class) Model class
(dict) Parameters
(dict) Values of all model variables

Parameters:	filename (str) – A path to save the pickled model

show_data(tables)¶

Sets model and module parameters from data prior to fit

Parameters:	tables (tflon.data.TableFeed) – Input tables used for this model training run

Tower API¶

class tflon.model.tower.Tower(params=None, use_gpu=False)¶

Tower is a slave class of the Model, which is designed to specifically abstract away the tensorflow interface of tflon.model.Model.

The combination of Tower and Model are effectively a hybrid class, since Model exposes its Tower instance methods directly via python missing property resolution (e.g __getattribute__).

The end user interacts with these methods and properties from within their Model instance as though they were attributes of Model.

add_input(name, shape=[None], dtype=tf.float32, sparse=False, ragged=False, ttype=<class 'tflon.data.tables.Table'>)¶

Add an input tensor. This method creates a dictionary alias name which can be used when feeding data to the model.

Input tensors can come in three varieties: dense (default), sparse, and ragged.

Parameters:	name (str) – The name reference for this tensor.

Keyword Parameters:: shape (iterable): The dense shape of this input tensor dtype (tf.type): The data type (e.g tf.float32) sparse (bool): Specify sparse tensor, sparse tensors are input via tf.SparseTensorValue ragged (bool): Specify a ragged array-type tensor, ragged tensor are input via tflon.data.RaggedTensorValue

Returns:	The input placeholder tensor
Return type:	tf.Tensor

add_loss(name, loss)¶

Add a named loss function to this model. All losses are summed to compute the final optimization target

Parameters:	name (str) – A name reference for this loss value loss (tensor-like) – The op calculating this loss

add_metric(name, metric)¶

Add a named metric op to this model.

Parameters:	name (str) – A name reference for this metric metric (tuple) – The op calculating this metric. This op should be able to handle aggregation of multiple batches. Tuples must have the form (eval_op, update_op), where eval_op computes the final metric value after multiple calls to update_op.

add_module(module, extra_parameters)¶

Add a module to this tower. This is automatically handled by the Module constructor.

Parameters:	module (tflon.toolkit.Module) – Instance of Module extra_parameters (dict) – A dictionary of parameter overrides for the module

add_output(name, out, transform=None)¶

Add an output tensor. Output tensors are returned by Model.infer

Optionally, output tensors can be transformed by a tflon.data.TensorTransform post-processing operation.

Parameters:	name (str) – A name for this output, used as a reference key in the dictionary returned by Model.infer out (tensor-like) – A tensor to fetch when writing output

Keyword Parameters:: transform (TensorTransform): A transformation to apply (default=None)

add_prior(loss)¶

Add a prior loss to this model.

Parameters:	loss (tensor-like) – The op calculating this loss

add_target(name, shape=[None], dtype=tf.float32, sparse=False, ragged=False, ttype=<class 'tflon.data.tables.Table'>)¶: Alias of Tower.add_input, differs only by semantic distinction.

biases¶: Get the collection of all biases for this model

featurize(batch)¶

Add each module’s features to the batch

Parameters:	batch (dict-like) – A batch dictionary containing data tables

feed(batch)¶

Given an input batch, construct a feed dictionary. This method handles mapping tables to support tensors for sparse and ragged inputs.

Returns:	of (placeholder, value) pairs, a valid feed_dict
Return type:	dict

feed_names(batch)¶

Given an input batch, construct a list of the names of all placeholder tensors required to feed data to the model. This will include names of support tensors for sparse and ragged inputs.

Returns:	A list of placeholder names
Return type:	list

get_bias(name, shape, dtype=tf.float32, initializer=None, prior=None, scope=None)¶

Get a bias variable. Biases are automatically added to tf.GraphKeys.TRAINABLE_VARIABLES, tf.GraphKeys.LOCAL_VARIABLES and tf.GraphKeys.BIASES

Parameters:	name (str) – The parameter name, which will result in a tensorflow variable referenced at scope/name:x shape (list or tuple) – The explicit shape of this variable

Keyword Parameters:: dtype (tf.dtype): The data type (default=tf.float32) initializer (tf.Operation): An initializer op (default=None, which gets this scope’s default initializer) scope (str): A scope for this var, used to select the appropriate default initializer for a module

get_module(name)¶

Get a module by name

Parameters:	name (str) – The module name

get_parameter(key, default=None, namespace=None)¶

Get the value of a parameter if it exists and is not None

Keyword Arguments:
Parameters:	key (str) – The parameter name
	namespace (str) – An optional namespace, searches for namespace/key instead of key (default=None)
Returns:	The value of the parameter
Raises:	`KeyError` – if the parameter key is not found or if its value is None

get_queue(**kwargs)¶: Construct a data queue for this tower, which can be used to pre-cache mini-batch data on the GPU in parallel.

get_variable(name, shape, dtype=tf.float32, initializer=<tensorflow.python.ops.init_ops.Zeros object>, trainable=False)¶

Get a tensorflow variable. Automatically selects the appropriate compute device.

Parameters:	name (str) – The parameter name, which will result in a tensorflow variable referenced at scope/name:x shape (list or tuple) – The explicit shape of this variable

Keyword Parameters:: dtype (tf.dtype): The data type (default=tf.float32) initializer (tf.Operation): An initializer op (default=tf.zeros_initializer()) trainable (bool): Whether to add to collection tf.GraphKeys.TRAINABLE_VARIABLES

get_weight(name, shape, dtype=tf.float32, initializer=None, prior=None, scope=None)¶

Get a weight variable. Weights are automatically added to tf.GraphKeys.TRAINABLE_VARIABLES, tf.GraphKeys.LOCAL_VARIABLES and tf.GraphKeys.WEIGHTS

Parameters:	name (str) – The parameter name, which will result in a tensorflow variable referenced at scope/name:x shape (list or tuple) – The explicit shape of this variable

Keyword Parameters:: dtype (tf.dtype): The data type (default=tf.float32) initializer (tf.Operation): An initializer op (default=None, which gets this scope’s default initializer) scope (str): A scope for this var, used to select the appropriate default initializer for a module

gradients¶

Return the gradients for all trainable variables.

Returns:	Tower gradients
Return type:	grads (list)

has_parameter(key, default=None, namespace=None)¶

Check if a parameter exists and is not None

Keyword Arguments:
Parameters:	key (str) – The parameter name
	namespace (str) – An optional namespace, searches for namespace/key instead of key (default=None)

inject_parameters()¶

Add the values of Model._parameters as class attributes. For a given (key, value) pair, this will add instance attributes of the form:

self.key = value

This behavior is currently optional, but will eventually become the default.

inputs¶: Get the input dictionary, (name, tf.Tensor) pairs. This does not necessarily contain feedable placeholders

inputs_and_targets¶: Get the joint dictionary of all inputs and targets

loader¶: Return the persistent tensor loader for this tower, which can be used to pre-cache data on the GPU.

loss¶: Get the aggregated loss op for this tower. Constructed as a sum of all named losses.

losses¶: Get a dictionary of all losses (name, tf.Tensor) pairs.

metric_variables¶: Get all variables associated with metric update operations

metrics¶: Get a dictionary of all metrics (name, (eval_op, update_op)) pairs.

modules¶: Get a copy of the modules dictionary, which consists of (module.name, module) pairs.

name¶: Get the name scope of the tower

outputs¶: Get the output dictionary, (name, tf.Tensor) pairs.

parameters¶: Return the parameter dictionary. This combines user-specified parameters, defaults in Model._parameters and all Module._parameters

phase¶

Boolean tensorflow variable specifying the current phase –

True = training phase
False = testing phase

placeholder_names¶: Get a dictionary of feedable placeholders to names. This will contain all inputs and targets, as well as support tensors for sparse and ragged inputs

placeholders¶: Get a dictionary of feedable placeholders. This will contain all inputs and targets, as well as support tensors for sparse and ragged inputs

populate(build_fn)¶

Bootstrap a tensorflow graph and perform sanity checks:

Inject parameters into the tower object attributes

Create the tensorflow ops associated with Model by calling build_fn.

Add additional ops joining all losses, or add a no_op if no losses

Perform sanity checks and emit warnings for missing elements (e.g no inputs, no losses)

Check for unused parameter values (often caused by a typo in a parameter name)

Parameters:	build_fn – A reference to Model._model()

schema¶: Return a tflon.data.Schema object implementing the model-specified schema (see Model._schema)

set_gradients(grads)¶

Set the gradients for trainable variables in this tower.

This is used to override the gradients property for use with optimizer-specific gradients.

set_loss(*keys)¶: Set the total loss op (Tower.loss) to the sum of the given named loss ops

set_parameter(key, value, namespace=None)¶

Set the value of a parameter

Keyword Arguments:
Parameters:	key (str) – The parameter name value – A parameter value (any dill-pickleable type is valid)
	namespace (str) – An optional namespace, searches for namespace/key instead of key (default=None)

set_parameter_default(key, value, namespace=None)¶

Set the value of a parameter if it is not already set

Keyword Arguments:
Parameters:	key (str) – The parameter name value – A parameter value (any dill-pickleable type is valid)
	namespace (str) – An optional namespace, searches for namespace/key instead of key (default=None)

set_training_phase(value)¶

Set the current phase. This is used by Model.infer, Model.fit and Model.evaluate

Values are:

True = training phase

False = testing phase

Parameters:	value (bool) – the new value of the phase variable

show_data(batch)¶

Show table data to each module attached to this tower. Used by Model.fit to set training parameters (e.g data normalization)

Parameters:	batch (dict-like) – A batch dictionary containing data tables

targets¶: Get the target dictionary, (name, tf.Tensor) pairs. This does not necessarily contain feedable placeholders

trainables¶: Get the collection of all trainables (weights + biases + other trainables) for this model

transforms¶: Get the transform dictionary, (name, TensorTransform) pairs.

vardict¶: Get a dictionary of all variables belonging to this tower

variables¶: Get the complete collection of all variables for this model (weights + biases + other trainbles + non-trainables)

weights¶: Get the collection of all weights for this model