org.bytedeco.javacpp

Class tensorflow.FixedUnigramCandidateSampler

java.lang.Object

org.bytedeco.javacpp.Pointer

org.bytedeco.javacpp.tensorflow.FixedUnigramCandidateSampler

All Implemented Interfaces:

AutoCloseable

Enclosing class:

tensorflow

@Namespace(value="tensorflow::ops")
 @NoOffset
public static class tensorflow.FixedUnigramCandidateSampler
extends Pointer

Generates labels for candidate sampling with a learned unigram distribution. A unigram sampler could use a fixed unigram distribution read from a file or passed in as an in-memory array instead of building up the distribution from data on the fly. There is also an option to skew the distribution by applying a distortion power to the weights. The vocabulary file should be in CSV-like format, with the last field being the weight associated with the word. For each batch, this op picks a single set of sampled candidate labels. The advantages of sampling candidates per-batch are simplicity and the possibility of efficient dense matrix multiplication. The disadvantage is that the sampled candidates must be chosen independently of the context and of the true labels. Arguments: * scope: A Scope object * true_classes: A batch_size * num_true matrix, in which each row contains the IDs of the num_true target_classes in the corresponding original label. * num_true: Number of true labels per context. * num_sampled: Number of candidates to randomly sample. * unique: If unique is true, we sample with rejection, so that all sampled candidates in a batch are unique. This requires some approximation to estimate the post-rejection sampling probabilities. * range_max: The sampler will sample integers from the interval [0, range_max). Optional attributes (see Attrs): * vocab_file: Each valid line in this file (which should have a CSV-like format) corresponds to a valid word ID. IDs are in sequential order, starting from num_reserved_ids. The last entry in each line is expected to be a value corresponding to the count or relative probability. Exactly one of vocab_file and unigrams needs to be passed to this op. * distortion: The distortion is used to skew the unigram probability distribution. Each weight is first raised to the distortion's power before adding to the internal unigram distribution. As a result, distortion = 1.0 gives regular unigram sampling (as defined by the vocab file), and distortion = 0.0 gives a uniform distribution. * num_reserved_ids: Optionally some reserved IDs can be added in the range [0, ..., num_reserved_ids) by the users. One use case is that a special unknown word token is used as ID 0. These IDs will have a sampling probability of 0. * num_shards: A sampler can be used to sample from a subset of the original range in order to speed up the whole computation through parallelism. This parameter (together with 'shard') indicates the number of partitions that are being used in the overall computation. * shard: A sampler can be used to sample from a subset of the original range in order to speed up the whole computation through parallelism. This parameter (together with 'num_shards') indicates the particular partition number of a sampler op, when partitioning is being used. * unigrams: A list of unigram counts or probabilities, one per ID in sequential order. Exactly one of vocab_file and unigrams should be passed to this op. * seed: If either seed or seed2 are set to be non-zero, the random number generator is seeded by the given seed. Otherwise, it is seeded by a random seed. * seed2: An second seed to avoid seed collision. Returns: * Output sampled_candidates: A vector of length num_sampled, in which each element is the ID of a sampled candidate. * Output true_expected_count: A batch_size * num_true matrix, representing the number of times each candidate is expected to occur in a batch of sampled candidates. If unique=true, then this is a probability. * Output sampled_expected_count: A vector of length num_sampled, for each sampled candidate representing the number of times the candidate is expected to occur in a batch of sampled candidates. If unique=true, then this is a probability.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	tensorflow.FixedUnigramCandidateSampler.Attrs Optional attribute setters for FixedUnigramCandidateSampler

Nested classes/interfaces inherited from class org.bytedeco.javacpp.Pointer

Pointer.CustomDeallocator, Pointer.Deallocator, Pointer.NativeDeallocator

Field Summary

Fields inherited from class org.bytedeco.javacpp.Pointer

address, capacity, limit, position

Constructor Summary

Constructors

Constructor and Description
FixedUnigramCandidateSampler(Pointer p) Pointer cast constructor.
FixedUnigramCandidateSampler(tensorflow.Scope scope, tensorflow.Input true_classes, long num_true, long num_sampled, boolean unique, long range_max)
FixedUnigramCandidateSampler(tensorflow.Scope scope, tensorflow.Input true_classes, long num_true, long num_sampled, boolean unique, long range_max, tensorflow.FixedUnigramCandidateSampler.Attrs attrs)

Method Summary

Modifier and Type	Method and Description
static tensorflow.FixedUnigramCandidateSampler.Attrs	Distortion(float x)
static tensorflow.FixedUnigramCandidateSampler.Attrs	NumReservedIds(long x)
static tensorflow.FixedUnigramCandidateSampler.Attrs	NumShards(long x)
tensorflow.Operation	operation()
tensorflow.FixedUnigramCandidateSampler	operation(tensorflow.Operation operation)
tensorflow.Output	sampled_candidates()
tensorflow.FixedUnigramCandidateSampler	sampled_candidates(tensorflow.Output sampled_candidates)
tensorflow.Output	sampled_expected_count()
tensorflow.FixedUnigramCandidateSampler	sampled_expected_count(tensorflow.Output sampled_expected_count)
static tensorflow.FixedUnigramCandidateSampler.Attrs	Seed(long x)
static tensorflow.FixedUnigramCandidateSampler.Attrs	Seed2(long x)
static tensorflow.FixedUnigramCandidateSampler.Attrs	Shard(long x)
tensorflow.Output	true_expected_count()
tensorflow.FixedUnigramCandidateSampler	true_expected_count(tensorflow.Output true_expected_count)
static tensorflow.FixedUnigramCandidateSampler.Attrs	Unigrams(float... x)
static tensorflow.FixedUnigramCandidateSampler.Attrs	Unigrams(FloatBuffer x)
static tensorflow.FixedUnigramCandidateSampler.Attrs	Unigrams(FloatPointer x)
static tensorflow.FixedUnigramCandidateSampler.Attrs	VocabFile(BytePointer x)
static tensorflow.FixedUnigramCandidateSampler.Attrs	VocabFile(String x)

Methods inherited from class org.bytedeco.javacpp.Pointer

address, asBuffer, asByteBuffer, availablePhysicalBytes, calloc, capacity, capacity, close, deallocate, deallocate, deallocateReferences, deallocator, deallocator, equals, fill, formatBytes, free, hashCode, isNull, limit, limit, malloc, maxBytes, maxPhysicalBytes, memchr, memcmp, memcpy, memmove, memset, offsetof, parseBytes, physicalBytes, position, position, put, realloc, setNull, sizeof, toString, totalBytes, totalPhysicalBytes, withDeallocator, zero

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

FixedUnigramCandidateSampler

public FixedUnigramCandidateSampler(Pointer p)

Pointer cast constructor. Invokes Pointer.Pointer(Pointer).

FixedUnigramCandidateSampler

public FixedUnigramCandidateSampler(@Const @ByRef
                                    tensorflow.Scope scope,
                                    @ByVal
                                    tensorflow.Input true_classes,
                                    @Cast(value="tensorflow::int64")
                                    long num_true,
                                    @Cast(value="tensorflow::int64")
                                    long num_sampled,
                                    @Cast(value="bool")
                                    boolean unique,
                                    @Cast(value="tensorflow::int64")
                                    long range_max)

FixedUnigramCandidateSampler

public FixedUnigramCandidateSampler(@Const @ByRef
                                    tensorflow.Scope scope,
                                    @ByVal
                                    tensorflow.Input true_classes,
                                    @Cast(value="tensorflow::int64")
                                    long num_true,
                                    @Cast(value="tensorflow::int64")
                                    long num_sampled,
                                    @Cast(value="bool")
                                    boolean unique,
                                    @Cast(value="tensorflow::int64")
                                    long range_max,
                                    @Const @ByRef
                                    tensorflow.FixedUnigramCandidateSampler.Attrs attrs)

Method Detail

VocabFile

@ByVal
public static tensorflow.FixedUnigramCandidateSampler.Attrs VocabFile(@tensorflow.StringPiece
                                                                              BytePointer x)

VocabFile

@ByVal
public static tensorflow.FixedUnigramCandidateSampler.Attrs VocabFile(@tensorflow.StringPiece
                                                                              String x)

Distortion

@ByVal
public static tensorflow.FixedUnigramCandidateSampler.Attrs Distortion(float x)

NumReservedIds

@ByVal
public static tensorflow.FixedUnigramCandidateSampler.Attrs NumReservedIds(@Cast(value="tensorflow::int64")
                                                                                   long x)

NumShards

@ByVal
public static tensorflow.FixedUnigramCandidateSampler.Attrs NumShards(@Cast(value="tensorflow::int64")
                                                                              long x)

Shard

@ByVal
public static tensorflow.FixedUnigramCandidateSampler.Attrs Shard(@Cast(value="tensorflow::int64")
                                                                          long x)

Unigrams

@ByVal
public static tensorflow.FixedUnigramCandidateSampler.Attrs Unigrams(@tensorflow.ArraySlice
                                                                             FloatPointer x)

Unigrams

@ByVal
public static tensorflow.FixedUnigramCandidateSampler.Attrs Unigrams(@tensorflow.ArraySlice
                                                                             FloatBuffer x)

Unigrams

@ByVal
public static tensorflow.FixedUnigramCandidateSampler.Attrs Unigrams(@tensorflow.ArraySlice
                                                                             float... x)

Seed

@ByVal
public static tensorflow.FixedUnigramCandidateSampler.Attrs Seed(@Cast(value="tensorflow::int64")
                                                                         long x)

Seed2

@ByVal
public static tensorflow.FixedUnigramCandidateSampler.Attrs Seed2(@Cast(value="tensorflow::int64")
                                                                          long x)

operation

@ByRef
public tensorflow.Operation operation()

operation

public tensorflow.FixedUnigramCandidateSampler operation(tensorflow.Operation operation)

sampled_candidates

@ByRef
public tensorflow.Output sampled_candidates()

sampled_candidates

public tensorflow.FixedUnigramCandidateSampler sampled_candidates(tensorflow.Output sampled_candidates)

true_expected_count

@ByRef
public tensorflow.Output true_expected_count()

true_expected_count

public tensorflow.FixedUnigramCandidateSampler true_expected_count(tensorflow.Output true_expected_count)

sampled_expected_count

@ByRef
public tensorflow.Output sampled_expected_count()

sampled_expected_count

public tensorflow.FixedUnigramCandidateSampler sampled_expected_count(tensorflow.Output sampled_expected_count)