robustintelligence

/

pi-t5gemma1b-v4.5

like 0

Follow

Robust Intelligence 92

---- Resolved TRT Profile ----
MIN_BATCH=1
OPT_BATCH=3
MAX_BATCH=12
MIN_SEQ_LEN=1
OPT_SEQ_LEN=512
MAX_SEQ_LEN=512
WORKSPACE_SIZE=24696061952
BUILDER_OPTIMIZATION_LEVEL=3
PRECISION=fp8

==== TensorRT Engine ====
Name: Unnamed Network 0 | Explicit Batch Engine

---- 2 Engine Input(s) ----
{input_ids [dtype=int64, shape=(-1, -1)],
attention_mask [dtype=int64, shape=(-1, -1)]}

---- 1 Engine Output(s) ----
{logits [dtype=float32, shape=(-1, 2)]}

---- Memory ----
Device Memory: 526706176 bytes

---- 1 Profile(s) (3 Tensor(s) Each) ----

• Profile: 0
  Tensor: input_ids (Input), Index: 0 | Shapes: min=(1, 1), opt=(3, 512), max=(12, 512)
  Tensor: attention_mask (Input), Index: 1 | Shapes: min=(1, 1), opt=(3, 512), max=(12, 512)
  Tensor: logits (Output), Index: 2 | Shape: (-1, 2)

---- 568 Layer(s) ----