gigl.common.beam.PassthroughCoder

class gigl.common.beam.coders.PassthroughCoder

Bases: Coder

Used as a dummy coder to just pass through the value without any special processing

Methods

__init__
as_deterministic_coder	Returns a deterministic version of self, if possible.
decode	Decodes the given byte string into the corresponding object.
decode_nested	Uses the underlying implementation to decode in nested format.
encode	Encodes the given object into a byte string.
encode_nested	Uses the underlying implementation to encode in nested format.
estimate_size	Estimates the encoded size of the given value, in bytes.
from_runner_api	Converts from an FunctionSpec to a Fn object.
from_type_hint
get_impl	For internal use only; no backwards-compatibility guarantees.
is_deterministic	Whether this coder is guaranteed to encode values deterministically.
is_kv_coder
key_coder
register_structured_urn	Register a coder that's completely defined by its urn and its component(s), if any, which are passed to construct the instance.
register_urn	Registers a urn with a constructor.
to_runner_api
to_runner_api_parameter
to_type_hint
value_coder

__eq__(other)

Return self==value.

__hash__()

Return hash(self).

__repr__()

Return repr(self).

__weakref__

list of weak references to the object (if defined)

as_deterministic_coder(step_label, error_message=None)

Returns a deterministic version of self, if possible.

Otherwise raises a value error.

decode(encoded)

Decodes the given byte string into the corresponding object.

decode_nested(encoded)

Uses the underlying implementation to decode in nested format.

encode(value: Any) → bytes

Encodes the given object into a byte string.

encode_nested(value)

Uses the underlying implementation to encode in nested format.

estimate_size(value)

Estimates the encoded size of the given value, in bytes.

Dataflow estimates the encoded size of a PCollection processed in a pipeline step by using the estimated size of a random sample of elements in that PCollection.

The default implementation encodes the given value and returns its byte size. If a coder can provide a fast estimate of the encoded size of a value (e.g., if the encoding has a fixed size), it can provide its estimate here to improve performance.

Arguments:

value: the value whose encoded size is to be estimated.

Returns:

The estimated encoded size of the given value.

classmethod from_runner_api(coder_proto: Type[CoderT], context: Coder) → CoderT

Converts from an FunctionSpec to a Fn object.

Prefer registering a urn with its parameter type and constructor.

get_impl()

For internal use only; no backwards-compatibility guarantees.

Returns the CoderImpl backing this Coder.

is_deterministic() → bool

Whether this coder is guaranteed to encode values deterministically.

A deterministic coder is required for key coders in GroupByKey operations to produce consistent results.

For example, note that the default coder, the PickleCoder, is not deterministic: the ordering of picked entries in maps may vary across executions since there is no defined order, and such a coder is not in general suitable for usage as a key coder in GroupByKey operations, since each instance of the same key may be encoded differently.

Returns:

Whether coder is deterministic.

static register_structured_urn(urn: str, cls: Type[Coder]) → None

Register a coder that’s completely defined by its urn and its component(s), if any, which are passed to construct the instance.

classmethod register_urn(urn, parameter_type, fn=None)

Registers a urn with a constructor.

For example, if ‘beam:fn:foo’ had parameter type FooPayload, one could write RunnerApiFn.register_urn(‘bean:fn:foo’, FooPayload, foo_from_proto) where foo_from_proto took as arguments a FooPayload and a PipelineContext. This function can also be used as a decorator rather than passing the callable in as the final parameter.

A corresponding to_runner_api_parameter method would be expected that returns the tuple (‘beam:fn:foo’, FooPayload)