&&&& RUNNING TensorRT.trtexec [TensorRT v8500] # trtexec --onnx=model.onnx --saveEngine=model.plan --minShapes=token_ids:1x128,attn_mask:1x128 --optShapes=token_ids:16x128,attn_mask:16x128 --maxShapes=token_ids:32x128,attn_mask:32x128 --verbose --workspace=14000 [05/12/2023-15:13:45] [I] === Model Options === [05/12/2023-15:13:45] [I] Format: ONNX [05/12/2023-15:13:45] [I] Model: model.onnx [05/12/2023-15:13:45] [I] Output: [05/12/2023-15:13:45] [I] === Build Options === [05/12/2023-15:13:45] [I] Max batch: explicit batch [05/12/2023-15:13:45] [I] Memory Pools: workspace: 14000 MiB, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default [05/12/2023-15:13:45] [I] minTiming: 1 [05/12/2023-15:13:45] [I] avgTiming: 8 [05/12/2023-15:13:45] [I] Precision: FP32 [05/12/2023-15:13:45] [I] LayerPrecisions: [05/12/2023-15:13:45] [I] Calibration: [05/12/2023-15:13:45] [I] Refit: Disabled [05/12/2023-15:13:45] [I] Sparsity: Disabled [05/12/2023-15:13:45] [I] Safe mode: Disabled [05/12/2023-15:13:45] [I] DirectIO mode: Disabled [05/12/2023-15:13:45] [I] Restricted mode: Disabled [05/12/2023-15:13:45] [I] Build only: Disabled [05/12/2023-15:13:45] [I] Save engine: model.plan [05/12/2023-15:13:45] [I] Load engine: [05/12/2023-15:13:45] [I] Profiling verbosity: 0 [05/12/2023-15:13:45] [I] Tactic sources: Using default tactic sources [05/12/2023-15:13:45] [I] timingCacheMode: local [05/12/2023-15:13:45] [I] timingCacheFile: [05/12/2023-15:13:45] [I] Heuristic: Disabled [05/12/2023-15:13:45] [I] Preview Features: Use default preview flags. [05/12/2023-15:13:45] [I] Input(s)s format: fp32:CHW [05/12/2023-15:13:45] [I] Output(s)s format: fp32:CHW [05/12/2023-15:13:45] [I] Input build shape: token_ids=1x128+16x128+32x128 [05/12/2023-15:13:45] [I] Input build shape: attn_mask=1x128+16x128+32x128 [05/12/2023-15:13:45] [I] Input calibration shapes: model [05/12/2023-15:13:45] [I] === System Options === [05/12/2023-15:13:45] [I] Device: 0 [05/12/2023-15:13:45] [I] DLACore: [05/12/2023-15:13:45] [I] Plugins: [05/12/2023-15:13:45] [I] === Inference Options === [05/12/2023-15:13:45] [I] Batch: Explicit [05/12/2023-15:13:45] [I] Input inference shape: attn_mask=16x128 [05/12/2023-15:13:45] [I] Input inference shape: token_ids=16x128 [05/12/2023-15:13:45] [I] Iterations: 10 [05/12/2023-15:13:45] [I] Duration: 3s (+ 200ms warm up) [05/12/2023-15:13:45] [I] Sleep time: 0ms [05/12/2023-15:13:45] [I] Idle time: 0ms [05/12/2023-15:13:45] [I] Streams: 1 [05/12/2023-15:13:45] [I] ExposeDMA: Disabled [05/12/2023-15:13:45] [I] Data transfers: Enabled [05/12/2023-15:13:45] [I] Spin-wait: Disabled [05/12/2023-15:13:45] [I] Multithreading: Disabled [05/12/2023-15:13:45] [I] CUDA Graph: Disabled [05/12/2023-15:13:45] [I] Separate profiling: Disabled [05/12/2023-15:13:45] [I] Time Deserialize: Disabled [05/12/2023-15:13:45] [I] Time Refit: Disabled [05/12/2023-15:13:45] [I] NVTX verbosity: 0 [05/12/2023-15:13:45] [I] Persistent Cache Ratio: 0 [05/12/2023-15:13:45] [I] Inputs: [05/12/2023-15:13:45] [I] === Reporting Options === [05/12/2023-15:13:45] [I] Verbose: Enabled [05/12/2023-15:13:45] [I] Averages: 10 inferences [05/12/2023-15:13:45] [I] Percentiles: 90,95,99 [05/12/2023-15:13:45] [I] Dump refittable layers:Disabled [05/12/2023-15:13:45] [I] Dump output: Disabled [05/12/2023-15:13:45] [I] Profile: Disabled [05/12/2023-15:13:45] [I] Export timing to JSON file: [05/12/2023-15:13:45] [I] Export output to JSON file: [05/12/2023-15:13:45] [I] Export profile to JSON file: [05/12/2023-15:13:45] [I] [05/12/2023-15:13:45] [I] === Device Information === [05/12/2023-15:13:45] [I] Selected Device: Tesla V100-SXM2-16GB [05/12/2023-15:13:45] [I] Compute Capability: 7.0 [05/12/2023-15:13:45] [I] SMs: 80 [05/12/2023-15:13:45] [I] Compute Clock Rate: 1.53 GHz [05/12/2023-15:13:45] [I] Device Global Memory: 16160 MiB [05/12/2023-15:13:45] [I] Shared Memory per SM: 96 KiB [05/12/2023-15:13:45] [I] Memory Bus Width: 4096 bits (ECC enabled) [05/12/2023-15:13:45] [I] Memory Clock Rate: 0.877 GHz [05/12/2023-15:13:45] [I] [05/12/2023-15:13:45] [I] TensorRT version: 8.5.0 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::BatchTilePlugin_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::CoordConvAC version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::CropAndResize version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::CropAndResizeDynamic version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::DecodeBbox3DPlugin version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::EfficientNMS_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::EfficientNMS_Explicit_TF_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::EfficientNMS_Implicit_TF_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::GenerateDetection_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::GridAnchorRect_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::MultilevelCropAndResize_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::MultilevelProposeROI_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::MultiscaleDeformableAttnPlugin_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::NMSDynamic_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::PillarScatterPlugin version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::Proposal version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::ProposalDynamic version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::Region_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::ScatterND version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::Split version 1 [05/12/2023-15:13:45] [V] [TRT] Registered plugin creator - ::VoxelGeneratorPlugin version 1 [05/12/2023-15:13:45] [I] [TRT] [MemUsageChange] Init CUDA: CPU +10, GPU +0, now: CPU 23, GPU 612 (MiB) [05/12/2023-15:13:48] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +171, GPU +46, now: CPU 247, GPU 658 (MiB) [05/12/2023-15:13:48] [V] [TRT] CUDA lazy loading is enabled. [05/12/2023-15:13:48] [I] Start parsing network model [05/12/2023-15:13:48] [I] [TRT] ---------------------------------------------------------------- [05/12/2023-15:13:48] [I] [TRT] Input filename: model.onnx [05/12/2023-15:13:48] [I] [TRT] ONNX IR version: 0.0.5 [05/12/2023-15:13:48] [I] [TRT] Opset version: 10 [05/12/2023-15:13:48] [I] [TRT] Producer name: pytorch [05/12/2023-15:13:48] [I] [TRT] Producer version: 1.13.0 [05/12/2023-15:13:48] [I] [TRT] Domain: [05/12/2023-15:13:48] [I] [TRT] Model version: 0 [05/12/2023-15:13:48] [I] [TRT] Doc string: [05/12/2023-15:13:48] [I] [TRT] ---------------------------------------------------------------- [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::BatchTilePlugin_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::BatchedNMSDynamic_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::CoordConvAC version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::CropAndResize version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::CropAndResizeDynamic version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::DecodeBbox3DPlugin version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::EfficientNMS_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::EfficientNMS_ONNX_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Explicit_TF_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Implicit_TF_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::GenerateDetection_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::GridAnchorRect_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::LReLU_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::MultilevelCropAndResize_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::MultilevelProposeROI_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::MultiscaleDeformableAttnPlugin_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::NMS_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::NMSDynamic_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::Normalize_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::PillarScatterPlugin version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::Proposal version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::ProposalDynamic version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::Region_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::Reorg_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::RPROI_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::ScatterND version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::Split version 1 [05/12/2023-15:13:48] [V] [TRT] Plugin creator already registered - ::VoxelGeneratorPlugin version 1 [05/12/2023-15:13:48] [V] [TRT] Adding network input: token_ids with dtype: int32, dimensions: (-1, -1) [05/12/2023-15:13:48] [V] [TRT] Registering tensor: token_ids for ONNX tensor: token_ids [05/12/2023-15:13:48] [V] [TRT] Adding network input: attn_mask with dtype: int32, dimensions: (-1, -1) [05/12/2023-15:13:48] [V] [TRT] Registering tensor: attn_mask for ONNX tensor: attn_mask [05/12/2023-15:13:48] [V] [TRT] Importing initializer: embeddings.position_ids [05/12/2023-15:13:48] [V] [TRT] Importing initializer: embeddings.word_embeddings.weight [05/12/2023-15:13:48] [V] [TRT] Importing initializer: embeddings.position_embeddings.weight [05/12/2023-15:13:48] [V] [TRT] Importing initializer: embeddings.token_type_embeddings.weight [05/12/2023-15:13:48] [V] [TRT] Importing initializer: embeddings.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Importing initializer: embeddings.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.0.attention.self.query.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.0.attention.self.key.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.0.attention.self.value.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.0.attention.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.0.attention.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.0.attention.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.0.intermediate.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.0.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.0.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.0.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.1.attention.self.query.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.1.attention.self.key.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.1.attention.self.value.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.1.attention.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.1.attention.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.1.attention.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.1.intermediate.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.1.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.1.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.1.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.2.attention.self.query.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.2.attention.self.key.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.2.attention.self.value.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.2.attention.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.2.attention.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.2.attention.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.2.intermediate.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.2.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.2.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.2.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.3.attention.self.query.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.3.attention.self.key.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.3.attention.self.value.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.3.attention.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.3.attention.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.3.attention.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.3.intermediate.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.3.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.3.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.3.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.4.attention.self.query.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.4.attention.self.key.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.4.attention.self.value.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.4.attention.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.4.attention.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.4.attention.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.4.intermediate.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.4.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.4.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.4.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.5.attention.self.query.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.5.attention.self.key.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.5.attention.self.value.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.5.attention.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.5.attention.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.5.attention.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.5.intermediate.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.5.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.5.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Importing initializer: encoder.layer.5.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: pooler.dense.weight [05/12/2023-15:13:48] [V] [TRT] Importing initializer: pooler.dense.bias [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_857 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_858 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_861 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_867 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_868 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_869 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_870 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_871 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_874 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_880 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_881 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_882 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_883 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_884 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_887 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_893 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_894 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_895 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_896 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_897 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_900 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_906 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_907 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_908 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_909 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_910 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_913 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_919 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_920 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_921 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_922 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_923 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_926 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_932 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_933 [05/12/2023-15:13:48] [V] [TRT] Importing initializer: onnx::MatMul_934 [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_0 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: token_ids [05/12/2023-15:13:48] [V] [TRT] Shape_0 [Shape] inputs: [token_ids -> (-1, -1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_0 for ONNX node: Shape_0 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_106 for ONNX tensor: onnx::Gather_106 [05/12/2023-15:13:48] [V] [TRT] Shape_0 [Shape] outputs: [onnx::Gather_106 -> (2)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_1 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_1 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_1 [Constant] outputs: [onnx::Gather_107 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_2 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_106 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_107 [05/12/2023-15:13:48] [V] [TRT] Gather_2 [Gather] inputs: [onnx::Gather_106 -> (2)[INT32]], [onnx::Gather_107 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_107 for ONNX node: onnx::Gather_107 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_2 for ONNX node: Gather_2 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_108 for ONNX tensor: onnx::Unsqueeze_108 [05/12/2023-15:13:48] [V] [TRT] Gather_2 [Gather] outputs: [onnx::Unsqueeze_108 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_3 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: token_ids [05/12/2023-15:13:48] [V] [TRT] Shape_3 [Shape] inputs: [token_ids -> (-1, -1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_3 for ONNX node: Shape_3 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_109 for ONNX tensor: onnx::Gather_109 [05/12/2023-15:13:48] [V] [TRT] Shape_3 [Shape] outputs: [onnx::Gather_109 -> (2)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_4 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_4 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_4 [Constant] outputs: [onnx::Gather_110 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_5 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_109 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_110 [05/12/2023-15:13:48] [V] [TRT] Gather_5 [Gather] inputs: [onnx::Gather_109 -> (2)[INT32]], [onnx::Gather_110 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_110 for ONNX node: onnx::Gather_110 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_5 for ONNX node: Gather_5 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_111 for ONNX tensor: onnx::Unsqueeze_111 [05/12/2023-15:13:48] [V] [TRT] Gather_5 [Gather] outputs: [onnx::Unsqueeze_111 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_6 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_6 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_6 [Constant] outputs: [onnx::Slice_112 -> (1, 512)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_7 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_7 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_7 [Constant] outputs: [onnx::Slice_853 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_8 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_111 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_8 [Unsqueeze] inputs: [onnx::Unsqueeze_111 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_8 for ONNX node: Unsqueeze_8 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Slice_116 for ONNX tensor: onnx::Slice_116 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_8 [Unsqueeze] outputs: [onnx::Slice_116 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_9 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_9 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_9 [Constant] outputs: [onnx::Slice_854 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_10 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_10 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_10 [Constant] outputs: [onnx::Slice_118 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Slice_11 [Slice] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Slice_112 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Slice_853 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Slice_116 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Slice_854 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Slice_118 [05/12/2023-15:13:48] [V] [TRT] Slice_11 [Slice] inputs: [onnx::Slice_112 -> (1, 512)[INT32]], [onnx::Slice_853 -> (1)[INT32]], [onnx::Slice_116 -> (1)[INT32]], [onnx::Slice_854 -> (1)[INT32]], [onnx::Slice_118 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Slice_112 for ONNX node: onnx::Slice_112 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Slice_11 for ONNX node: Slice_11 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Expand_119 for ONNX tensor: onnx::Expand_119 [05/12/2023-15:13:48] [V] [TRT] Slice_11 [Slice] outputs: [onnx::Expand_119 -> (1, -1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_12 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_108 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_12 [Unsqueeze] inputs: [onnx::Unsqueeze_108 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_12 for ONNX node: Unsqueeze_12 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_120 for ONNX tensor: onnx::Concat_120 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_12 [Unsqueeze] outputs: [onnx::Concat_120 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_13 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_111 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_13 [Unsqueeze] inputs: [onnx::Unsqueeze_111 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_13 for ONNX node: Unsqueeze_13 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_121 for ONNX tensor: onnx::Concat_121 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_13 [Unsqueeze] outputs: [onnx::Concat_121 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Concat_14 [Concat] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_120 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_121 [05/12/2023-15:13:48] [V] [TRT] Concat_14 [Concat] inputs: [onnx::Concat_120 -> (1)[INT32]], [onnx::Concat_121 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Concat_14 for ONNX node: Concat_14 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Reshape_122 for ONNX tensor: onnx::Reshape_122 [05/12/2023-15:13:48] [V] [TRT] Concat_14 [Concat] outputs: [onnx::Reshape_122 -> (2)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_15 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_15 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_15 [Constant] outputs: [onnx::Reshape_123 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Reshape_16 [Reshape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Reshape_122 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Reshape_123 [05/12/2023-15:13:48] [V] [TRT] Reshape_16 [Reshape] inputs: [onnx::Reshape_122 -> (2)[INT32]], [onnx::Reshape_123 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Reshape_16 for ONNX node: Reshape_16 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Shape_124 for ONNX tensor: onnx::Shape_124 [05/12/2023-15:13:48] [V] [TRT] Reshape_16 [Reshape] outputs: [onnx::Shape_124 -> (2)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_17 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_124 [05/12/2023-15:13:48] [V] [TRT] Shape_17 [Shape] inputs: [onnx::Shape_124 -> (2)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_17 for ONNX node: Shape_17 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::ConstantOfShape_125 for ONNX tensor: onnx::ConstantOfShape_125 [05/12/2023-15:13:48] [V] [TRT] Shape_17 [Shape] outputs: [onnx::ConstantOfShape_125 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: ConstantOfShape_18 [ConstantOfShape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::ConstantOfShape_125 [05/12/2023-15:13:48] [V] [TRT] ConstantOfShape_18 [ConstantOfShape] inputs: [onnx::ConstantOfShape_125 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: ConstantOfShape_18 for ONNX node: ConstantOfShape_18 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Mul_126 for ONNX tensor: onnx::Mul_126 [05/12/2023-15:13:48] [V] [TRT] ConstantOfShape_18 [ConstantOfShape] outputs: [onnx::Mul_126 -> (2)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_19 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_19 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_19 [Constant] outputs: [onnx::Mul_127 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Mul_20 [Mul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Mul_126 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Mul_127 [05/12/2023-15:13:48] [V] [TRT] Mul_20 [Mul] inputs: [onnx::Mul_126 -> (2)[INT32]], [onnx::Mul_127 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Mul_127 for ONNX node: onnx::Mul_127 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Mul_20 for ONNX node: Mul_20 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Equal_128 for ONNX tensor: onnx::Equal_128 [05/12/2023-15:13:48] [V] [TRT] Mul_20 [Mul] outputs: [onnx::Equal_128 -> (2)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Equal_21 [Equal] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_124 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Equal_128 [05/12/2023-15:13:48] [V] [TRT] Equal_21 [Equal] inputs: [onnx::Shape_124 -> (2)[INT32]], [onnx::Equal_128 -> (2)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Equal_21 for ONNX node: Equal_21 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Where_129 for ONNX tensor: onnx::Where_129 [05/12/2023-15:13:48] [V] [TRT] Equal_21 [Equal] outputs: [onnx::Where_129 -> (2)[BOOL]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Where_22 [Where] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Where_129 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Mul_126 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_124 [05/12/2023-15:13:48] [V] [TRT] Where_22 [Where] inputs: [onnx::Where_129 -> (2)[BOOL]], [onnx::Mul_126 -> (2)[INT32]], [onnx::Shape_124 -> (2)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Where_22 for ONNX node: Where_22 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Expand_130 for ONNX tensor: onnx::Expand_130 [05/12/2023-15:13:48] [V] [TRT] Where_22 [Where] outputs: [onnx::Expand_130 -> (2)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Expand_23 [Expand] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Expand_119 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Expand_130 [05/12/2023-15:13:48] [V] [TRT] Expand_23 [Expand] inputs: [onnx::Expand_119 -> (1, -1)[INT32]], [onnx::Expand_130 -> (2)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Expand_23 for ONNX node: Expand_23 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: input for ONNX tensor: input [05/12/2023-15:13:48] [V] [TRT] Expand_23 [Expand] outputs: [input -> (-1, -1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_24 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: attn_mask [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_24 [Unsqueeze] inputs: [attn_mask -> (-1, -1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_24 for ONNX node: Unsqueeze_24 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_132 for ONNX tensor: onnx::Unsqueeze_132 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_24 [Unsqueeze] outputs: [onnx::Unsqueeze_132 -> (-1, 1, -1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_25 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_132 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_25 [Unsqueeze] inputs: [onnx::Unsqueeze_132 -> (-1, 1, -1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (_, 1, _), unsqueezing to: (_, _, _, _) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_25 for ONNX node: Unsqueeze_25 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Cast_133 for ONNX tensor: onnx::Cast_133 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_25 [Unsqueeze] outputs: [onnx::Cast_133 -> (-1, 1, 1, -1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Cast_26 [Cast] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Cast_133 [05/12/2023-15:13:48] [V] [TRT] Cast_26 [Cast] inputs: [onnx::Cast_133 -> (-1, 1, 1, -1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Casting to type: float32 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Cast_26 for ONNX node: Cast_26 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Sub_134 for ONNX tensor: onnx::Sub_134 [05/12/2023-15:13:48] [V] [TRT] Cast_26 [Cast] outputs: [onnx::Sub_134 -> (-1, 1, 1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_27 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_27 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_27 [Constant] outputs: [onnx::Sub_135 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Sub_28 [Sub] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Sub_135 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Sub_134 [05/12/2023-15:13:48] [V] [TRT] Sub_28 [Sub] inputs: [onnx::Sub_135 -> ()[FLOAT]], [onnx::Sub_134 -> (-1, 1, 1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Sub_135 for ONNX node: onnx::Sub_135 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Sub_28 for ONNX node: Sub_28 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Mul_136 for ONNX tensor: onnx::Mul_136 [05/12/2023-15:13:48] [V] [TRT] Sub_28 [Sub] outputs: [onnx::Mul_136 -> (-1, 1, 1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_29 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_29 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_29 [Constant] outputs: [onnx::Mul_137 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Mul_30 [Mul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Mul_136 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Mul_137 [05/12/2023-15:13:48] [V] [TRT] Mul_30 [Mul] inputs: [onnx::Mul_136 -> (-1, 1, 1, -1)[FLOAT]], [onnx::Mul_137 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Mul_137 for ONNX node: onnx::Mul_137 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Mul_30 for ONNX node: Mul_30 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_138 for ONNX tensor: onnx::Add_138 [05/12/2023-15:13:48] [V] [TRT] Mul_30 [Mul] outputs: [onnx::Add_138 -> (-1, 1, 1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_31 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: token_ids [05/12/2023-15:13:48] [V] [TRT] Shape_31 [Shape] inputs: [token_ids -> (-1, -1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_31 for ONNX node: Shape_31 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_139 for ONNX tensor: onnx::Gather_139 [05/12/2023-15:13:48] [V] [TRT] Shape_31 [Shape] outputs: [onnx::Gather_139 -> (2)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_32 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_32 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_32 [Constant] outputs: [onnx::Gather_140 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_33 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_139 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_140 [05/12/2023-15:13:48] [V] [TRT] Gather_33 [Gather] inputs: [onnx::Gather_139 -> (2)[INT32]], [onnx::Gather_140 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_140 for ONNX node: onnx::Gather_140 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_33 for ONNX node: Gather_33 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_141 for ONNX tensor: onnx::Unsqueeze_141 [05/12/2023-15:13:48] [V] [TRT] Gather_33 [Gather] outputs: [onnx::Unsqueeze_141 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_34 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_34 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_34 [Constant] outputs: [onnx::Slice_855 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_35 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_141 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_35 [Unsqueeze] inputs: [onnx::Unsqueeze_141 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_35 for ONNX node: Unsqueeze_35 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Slice_145 for ONNX tensor: onnx::Slice_145 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_35 [Unsqueeze] outputs: [onnx::Slice_145 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_36 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_36 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_36 [Constant] outputs: [onnx::Slice_856 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_37 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_37 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_37 [Constant] outputs: [onnx::Slice_147 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Slice_38 [Slice] [05/12/2023-15:13:48] [V] [TRT] Searching for input: embeddings.position_ids [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Slice_855 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Slice_145 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Slice_856 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Slice_147 [05/12/2023-15:13:48] [V] [TRT] Slice_38 [Slice] inputs: [embeddings.position_ids -> (1, 512)[INT32]], [onnx::Slice_855 -> (1)[INT32]], [onnx::Slice_145 -> (1)[INT32]], [onnx::Slice_856 -> (1)[INT32]], [onnx::Slice_147 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: embeddings.position_ids for ONNX node: embeddings.position_ids [05/12/2023-15:13:48] [V] [TRT] Registering layer: Slice_38 for ONNX node: Slice_38 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: input.4 for ONNX tensor: input.4 [05/12/2023-15:13:48] [V] [TRT] Slice_38 [Slice] outputs: [input.4 -> (1, -1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_39 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: embeddings.word_embeddings.weight [05/12/2023-15:13:48] [V] [TRT] Searching for input: token_ids [05/12/2023-15:13:48] [V] [TRT] Gather_39 [Gather] inputs: [embeddings.word_embeddings.weight -> (30522, 384)[FLOAT]], [token_ids -> (-1, -1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: embeddings.word_embeddings.weight for ONNX node: embeddings.word_embeddings.weight [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_39 for ONNX node: Gather_39 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_149 for ONNX tensor: onnx::Add_149 [05/12/2023-15:13:48] [V] [TRT] Gather_39 [Gather] outputs: [onnx::Add_149 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_40 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: embeddings.token_type_embeddings.weight [05/12/2023-15:13:48] [V] [TRT] Searching for input: input [05/12/2023-15:13:48] [V] [TRT] Gather_40 [Gather] inputs: [embeddings.token_type_embeddings.weight -> (2, 384)[FLOAT]], [input -> (-1, -1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: embeddings.token_type_embeddings.weight for ONNX node: embeddings.token_type_embeddings.weight [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_40 for ONNX node: Gather_40 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_150 for ONNX tensor: onnx::Add_150 [05/12/2023-15:13:48] [V] [TRT] Gather_40 [Gather] outputs: [onnx::Add_150 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_41 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_149 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_150 [05/12/2023-15:13:48] [V] [TRT] Add_41 [Add] inputs: [onnx::Add_149 -> (-1, -1, 384)[FLOAT]], [onnx::Add_150 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_41 for ONNX node: Add_41 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_151 for ONNX tensor: onnx::Add_151 [05/12/2023-15:13:48] [V] [TRT] Add_41 [Add] outputs: [onnx::Add_151 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_42 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: embeddings.position_embeddings.weight [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.4 [05/12/2023-15:13:48] [V] [TRT] Gather_42 [Gather] inputs: [embeddings.position_embeddings.weight -> (512, 384)[FLOAT]], [input.4 -> (1, -1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: embeddings.position_embeddings.weight for ONNX node: embeddings.position_embeddings.weight [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_42 for ONNX node: Gather_42 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_152 for ONNX tensor: onnx::Add_152 [05/12/2023-15:13:48] [V] [TRT] Gather_42 [Gather] outputs: [onnx::Add_152 -> (1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_43 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_151 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_152 [05/12/2023-15:13:48] [V] [TRT] Add_43 [Add] inputs: [onnx::Add_151 -> (-1, -1, 384)[FLOAT]], [onnx::Add_152 -> (1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_43 for ONNX node: Add_43 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::ReduceMean_153 for ONNX tensor: onnx::ReduceMean_153 [05/12/2023-15:13:48] [V] [TRT] Add_43 [Add] outputs: [onnx::ReduceMean_153 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: ReduceMean_44 [ReduceMean] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::ReduceMean_153 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_44 [ReduceMean] inputs: [onnx::ReduceMean_153 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: ReduceMean_44 for ONNX node: ReduceMean_44 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Sub_154 for ONNX tensor: onnx::Sub_154 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_44 [ReduceMean] outputs: [onnx::Sub_154 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Sub_45 [Sub] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::ReduceMean_153 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Sub_154 [05/12/2023-15:13:48] [V] [TRT] Sub_45 [Sub] inputs: [onnx::ReduceMean_153 -> (-1, -1, 384)[FLOAT]], [onnx::Sub_154 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Sub_45 for ONNX node: Sub_45 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Pow_155 for ONNX tensor: onnx::Pow_155 [05/12/2023-15:13:48] [V] [TRT] Sub_45 [Sub] outputs: [onnx::Pow_155 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_46 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_46 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_46 [Constant] outputs: [onnx::Pow_156 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Pow_47 [Pow] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Pow_155 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Pow_156 [05/12/2023-15:13:48] [V] [TRT] Pow_47 [Pow] inputs: [onnx::Pow_155 -> (-1, -1, 384)[FLOAT]], [onnx::Pow_156 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Pow_156 for ONNX node: onnx::Pow_156 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Pow_47 for ONNX node: Pow_47 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::ReduceMean_157 for ONNX tensor: onnx::ReduceMean_157 [05/12/2023-15:13:48] [V] [TRT] Pow_47 [Pow] outputs: [onnx::ReduceMean_157 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: ReduceMean_48 [ReduceMean] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::ReduceMean_157 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_48 [ReduceMean] inputs: [onnx::ReduceMean_157 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: ReduceMean_48 for ONNX node: ReduceMean_48 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_158 for ONNX tensor: onnx::Add_158 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_48 [ReduceMean] outputs: [onnx::Add_158 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_49 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_49 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_49 [Constant] outputs: [onnx::Add_159 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_50 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_158 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_159 [05/12/2023-15:13:48] [V] [TRT] Add_50 [Add] inputs: [onnx::Add_158 -> (-1, -1, 1)[FLOAT]], [onnx::Add_159 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Add_159 for ONNX node: onnx::Add_159 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_50 for ONNX node: Add_50 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Sqrt_160 for ONNX tensor: onnx::Sqrt_160 [05/12/2023-15:13:48] [V] [TRT] Add_50 [Add] outputs: [onnx::Sqrt_160 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Sqrt_51 [Sqrt] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Sqrt_160 [05/12/2023-15:13:48] [V] [TRT] Sqrt_51 [Sqrt] inputs: [onnx::Sqrt_160 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Sqrt_51 for ONNX node: Sqrt_51 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Div_161 for ONNX tensor: onnx::Div_161 [05/12/2023-15:13:48] [V] [TRT] Sqrt_51 [Sqrt] outputs: [onnx::Div_161 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Div_52 [Div] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Pow_155 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Div_161 [05/12/2023-15:13:48] [V] [TRT] Div_52 [Div] inputs: [onnx::Pow_155 -> (-1, -1, 384)[FLOAT]], [onnx::Div_161 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Div_52 for ONNX node: Div_52 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Mul_162 for ONNX tensor: onnx::Mul_162 [05/12/2023-15:13:48] [V] [TRT] Div_52 [Div] outputs: [onnx::Mul_162 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Mul_53 [Mul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Mul_162 [05/12/2023-15:13:48] [V] [TRT] Searching for input: embeddings.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Mul_53 [Mul] inputs: [onnx::Mul_162 -> (-1, -1, 384)[FLOAT]], [embeddings.LayerNorm.weight -> (384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: embeddings.LayerNorm.weight for ONNX node: embeddings.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Registering layer: Mul_53 for ONNX node: Mul_53 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_163 for ONNX tensor: onnx::Add_163 [05/12/2023-15:13:48] [V] [TRT] Mul_53 [Mul] outputs: [onnx::Add_163 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_54 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_163 [05/12/2023-15:13:48] [V] [TRT] Searching for input: embeddings.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Add_54 [Add] inputs: [onnx::Add_163 -> (-1, -1, 384)[FLOAT]], [embeddings.LayerNorm.bias -> (384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: embeddings.LayerNorm.bias for ONNX node: embeddings.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_54 for ONNX node: Add_54 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: input.8 for ONNX tensor: input.8 [05/12/2023-15:13:48] [V] [TRT] Add_54 [Add] outputs: [input.8 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_55 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.8 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_857 [05/12/2023-15:13:48] [V] [TRT] MatMul_55 [MatMul] inputs: [input.8 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_857 -> (384, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::MatMul_857 for ONNX node: onnx::MatMul_857 [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_55 for ONNX node: MatMul_55 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_166 for ONNX tensor: onnx::Add_166 [05/12/2023-15:13:48] [V] [TRT] MatMul_55 [MatMul] outputs: [onnx::Add_166 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_56 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.0.attention.self.query.bias [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_166 [05/12/2023-15:13:48] [V] [TRT] Add_56 [Add] inputs: [encoder.layer.0.attention.self.query.bias -> (384)[FLOAT]], [onnx::Add_166 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.0.attention.self.query.bias for ONNX node: encoder.layer.0.attention.self.query.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_56 for ONNX node: Add_56 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: mixed_query_layer for ONNX tensor: mixed_query_layer [05/12/2023-15:13:48] [V] [TRT] Add_56 [Add] outputs: [mixed_query_layer -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_57 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.8 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_858 [05/12/2023-15:13:48] [V] [TRT] MatMul_57 [MatMul] inputs: [input.8 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_858 -> (384, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::MatMul_858 for ONNX node: onnx::MatMul_858 [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_57 for ONNX node: MatMul_57 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_169 for ONNX tensor: onnx::Add_169 [05/12/2023-15:13:48] [V] [TRT] MatMul_57 [MatMul] outputs: [onnx::Add_169 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_58 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.0.attention.self.key.bias [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_169 [05/12/2023-15:13:48] [V] [TRT] Add_58 [Add] inputs: [encoder.layer.0.attention.self.key.bias -> (384)[FLOAT]], [onnx::Add_169 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.0.attention.self.key.bias for ONNX node: encoder.layer.0.attention.self.key.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_58 for ONNX node: Add_58 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Shape_170 for ONNX tensor: onnx::Shape_170 [05/12/2023-15:13:48] [V] [TRT] Add_58 [Add] outputs: [onnx::Shape_170 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_59 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_170 [05/12/2023-15:13:48] [V] [TRT] Shape_59 [Shape] inputs: [onnx::Shape_170 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_59 for ONNX node: Shape_59 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_171 for ONNX tensor: onnx::Gather_171 [05/12/2023-15:13:48] [V] [TRT] Shape_59 [Shape] outputs: [onnx::Gather_171 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_60 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_60 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_60 [Constant] outputs: [onnx::Gather_172 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_61 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_171 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_172 [05/12/2023-15:13:48] [V] [TRT] Gather_61 [Gather] inputs: [onnx::Gather_171 -> (3)[INT32]], [onnx::Gather_172 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_172 for ONNX node: onnx::Gather_172 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_61 for ONNX node: Gather_61 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_173 for ONNX tensor: onnx::Unsqueeze_173 [05/12/2023-15:13:48] [V] [TRT] Gather_61 [Gather] outputs: [onnx::Unsqueeze_173 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_62 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_170 [05/12/2023-15:13:48] [V] [TRT] Shape_62 [Shape] inputs: [onnx::Shape_170 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_62 for ONNX node: Shape_62 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_174 for ONNX tensor: onnx::Gather_174 [05/12/2023-15:13:48] [V] [TRT] Shape_62 [Shape] outputs: [onnx::Gather_174 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_63 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_63 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_63 [Constant] outputs: [onnx::Gather_175 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_64 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_174 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_175 [05/12/2023-15:13:48] [V] [TRT] Gather_64 [Gather] inputs: [onnx::Gather_174 -> (3)[INT32]], [onnx::Gather_175 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_175 for ONNX node: onnx::Gather_175 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_64 for ONNX node: Gather_64 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_176 for ONNX tensor: onnx::Unsqueeze_176 [05/12/2023-15:13:48] [V] [TRT] Gather_64 [Gather] outputs: [onnx::Unsqueeze_176 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_65 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_173 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_65 [Unsqueeze] inputs: [onnx::Unsqueeze_173 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_65 for ONNX node: Unsqueeze_65 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_179 for ONNX tensor: onnx::Concat_179 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_65 [Unsqueeze] outputs: [onnx::Concat_179 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_66 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_176 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_66 [Unsqueeze] inputs: [onnx::Unsqueeze_176 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_66 for ONNX node: Unsqueeze_66 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_180 for ONNX tensor: onnx::Concat_180 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_66 [Unsqueeze] outputs: [onnx::Concat_180 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_67 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_67 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_67 [Constant] outputs: [onnx::Concat_859 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_68 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_68 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_68 [Constant] outputs: [onnx::Concat_860 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Concat_69 [Concat] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_179 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_180 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_859 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_860 [05/12/2023-15:13:48] [V] [TRT] Concat_69 [Concat] inputs: [onnx::Concat_179 -> (1)[INT32]], [onnx::Concat_180 -> (1)[INT32]], [onnx::Concat_859 -> (1)[INT32]], [onnx::Concat_860 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_859 for ONNX node: onnx::Concat_859 [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_860 for ONNX node: onnx::Concat_860 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Concat_69 for ONNX node: Concat_69 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Reshape_183 for ONNX tensor: onnx::Reshape_183 [05/12/2023-15:13:48] [V] [TRT] Concat_69 [Concat] outputs: [onnx::Reshape_183 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Reshape_70 [Reshape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_170 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Reshape_183 [05/12/2023-15:13:48] [V] [TRT] Reshape_70 [Reshape] inputs: [onnx::Shape_170 -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_183 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Reshape_70 for ONNX node: Reshape_70 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Transpose_184 for ONNX tensor: onnx::Transpose_184 [05/12/2023-15:13:48] [V] [TRT] Reshape_70 [Reshape] outputs: [onnx::Transpose_184 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_71 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.8 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_861 [05/12/2023-15:13:48] [V] [TRT] MatMul_71 [MatMul] inputs: [input.8 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_861 -> (384, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::MatMul_861 for ONNX node: onnx::MatMul_861 [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_71 for ONNX node: MatMul_71 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_186 for ONNX tensor: onnx::Add_186 [05/12/2023-15:13:48] [V] [TRT] MatMul_71 [MatMul] outputs: [onnx::Add_186 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_72 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.0.attention.self.value.bias [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_186 [05/12/2023-15:13:48] [V] [TRT] Add_72 [Add] inputs: [encoder.layer.0.attention.self.value.bias -> (384)[FLOAT]], [onnx::Add_186 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.0.attention.self.value.bias for ONNX node: encoder.layer.0.attention.self.value.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_72 for ONNX node: Add_72 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Shape_187 for ONNX tensor: onnx::Shape_187 [05/12/2023-15:13:48] [V] [TRT] Add_72 [Add] outputs: [onnx::Shape_187 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_73 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_187 [05/12/2023-15:13:48] [V] [TRT] Shape_73 [Shape] inputs: [onnx::Shape_187 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_73 for ONNX node: Shape_73 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_188 for ONNX tensor: onnx::Gather_188 [05/12/2023-15:13:48] [V] [TRT] Shape_73 [Shape] outputs: [onnx::Gather_188 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_74 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_74 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_74 [Constant] outputs: [onnx::Gather_189 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_75 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_188 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_189 [05/12/2023-15:13:48] [V] [TRT] Gather_75 [Gather] inputs: [onnx::Gather_188 -> (3)[INT32]], [onnx::Gather_189 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_189 for ONNX node: onnx::Gather_189 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_75 for ONNX node: Gather_75 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_190 for ONNX tensor: onnx::Unsqueeze_190 [05/12/2023-15:13:48] [V] [TRT] Gather_75 [Gather] outputs: [onnx::Unsqueeze_190 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_76 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_187 [05/12/2023-15:13:48] [V] [TRT] Shape_76 [Shape] inputs: [onnx::Shape_187 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_76 for ONNX node: Shape_76 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_191 for ONNX tensor: onnx::Gather_191 [05/12/2023-15:13:48] [V] [TRT] Shape_76 [Shape] outputs: [onnx::Gather_191 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_77 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_77 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_77 [Constant] outputs: [onnx::Gather_192 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_78 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_191 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_192 [05/12/2023-15:13:48] [V] [TRT] Gather_78 [Gather] inputs: [onnx::Gather_191 -> (3)[INT32]], [onnx::Gather_192 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_192 for ONNX node: onnx::Gather_192 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_78 for ONNX node: Gather_78 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_193 for ONNX tensor: onnx::Unsqueeze_193 [05/12/2023-15:13:48] [V] [TRT] Gather_78 [Gather] outputs: [onnx::Unsqueeze_193 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_79 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_190 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_79 [Unsqueeze] inputs: [onnx::Unsqueeze_190 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_79 for ONNX node: Unsqueeze_79 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_196 for ONNX tensor: onnx::Concat_196 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_79 [Unsqueeze] outputs: [onnx::Concat_196 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_80 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_193 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_80 [Unsqueeze] inputs: [onnx::Unsqueeze_193 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_80 for ONNX node: Unsqueeze_80 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_197 for ONNX tensor: onnx::Concat_197 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_80 [Unsqueeze] outputs: [onnx::Concat_197 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_81 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_81 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_81 [Constant] outputs: [onnx::Concat_862 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_82 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_82 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_82 [Constant] outputs: [onnx::Concat_863 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Concat_83 [Concat] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_196 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_197 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_862 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_863 [05/12/2023-15:13:48] [V] [TRT] Concat_83 [Concat] inputs: [onnx::Concat_196 -> (1)[INT32]], [onnx::Concat_197 -> (1)[INT32]], [onnx::Concat_862 -> (1)[INT32]], [onnx::Concat_863 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_862 for ONNX node: onnx::Concat_862 [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_863 for ONNX node: onnx::Concat_863 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Concat_83 for ONNX node: Concat_83 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Reshape_200 for ONNX tensor: onnx::Reshape_200 [05/12/2023-15:13:48] [V] [TRT] Concat_83 [Concat] outputs: [onnx::Reshape_200 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Reshape_84 [Reshape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_187 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Reshape_200 [05/12/2023-15:13:48] [V] [TRT] Reshape_84 [Reshape] inputs: [onnx::Shape_187 -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_200 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Reshape_84 for ONNX node: Reshape_84 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Transpose_201 for ONNX tensor: onnx::Transpose_201 [05/12/2023-15:13:48] [V] [TRT] Reshape_84 [Reshape] outputs: [onnx::Transpose_201 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Transpose_85 [Transpose] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Transpose_201 [05/12/2023-15:13:48] [V] [TRT] Transpose_85 [Transpose] inputs: [onnx::Transpose_201 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Transpose_85 for ONNX node: Transpose_85 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::MatMul_202 for ONNX tensor: onnx::MatMul_202 [05/12/2023-15:13:48] [V] [TRT] Transpose_85 [Transpose] outputs: [onnx::MatMul_202 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_86 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: mixed_query_layer [05/12/2023-15:13:48] [V] [TRT] Shape_86 [Shape] inputs: [mixed_query_layer -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_86 for ONNX node: Shape_86 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_203 for ONNX tensor: onnx::Gather_203 [05/12/2023-15:13:48] [V] [TRT] Shape_86 [Shape] outputs: [onnx::Gather_203 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_87 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_87 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_87 [Constant] outputs: [onnx::Gather_204 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_88 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_203 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_204 [05/12/2023-15:13:48] [V] [TRT] Gather_88 [Gather] inputs: [onnx::Gather_203 -> (3)[INT32]], [onnx::Gather_204 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_204 for ONNX node: onnx::Gather_204 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_88 for ONNX node: Gather_88 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_205 for ONNX tensor: onnx::Unsqueeze_205 [05/12/2023-15:13:48] [V] [TRT] Gather_88 [Gather] outputs: [onnx::Unsqueeze_205 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_89 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: mixed_query_layer [05/12/2023-15:13:48] [V] [TRT] Shape_89 [Shape] inputs: [mixed_query_layer -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_89 for ONNX node: Shape_89 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_206 for ONNX tensor: onnx::Gather_206 [05/12/2023-15:13:48] [V] [TRT] Shape_89 [Shape] outputs: [onnx::Gather_206 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_90 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_90 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_90 [Constant] outputs: [onnx::Gather_207 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_91 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_206 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_207 [05/12/2023-15:13:48] [V] [TRT] Gather_91 [Gather] inputs: [onnx::Gather_206 -> (3)[INT32]], [onnx::Gather_207 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_207 for ONNX node: onnx::Gather_207 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_91 for ONNX node: Gather_91 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_208 for ONNX tensor: onnx::Unsqueeze_208 [05/12/2023-15:13:48] [V] [TRT] Gather_91 [Gather] outputs: [onnx::Unsqueeze_208 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_92 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_205 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_92 [Unsqueeze] inputs: [onnx::Unsqueeze_205 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_92 for ONNX node: Unsqueeze_92 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_211 for ONNX tensor: onnx::Concat_211 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_92 [Unsqueeze] outputs: [onnx::Concat_211 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_93 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_208 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_93 [Unsqueeze] inputs: [onnx::Unsqueeze_208 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_93 for ONNX node: Unsqueeze_93 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_212 for ONNX tensor: onnx::Concat_212 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_93 [Unsqueeze] outputs: [onnx::Concat_212 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_94 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_94 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_94 [Constant] outputs: [onnx::Concat_864 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_95 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_95 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_95 [Constant] outputs: [onnx::Concat_865 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Concat_96 [Concat] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_211 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_212 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_864 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_865 [05/12/2023-15:13:48] [V] [TRT] Concat_96 [Concat] inputs: [onnx::Concat_211 -> (1)[INT32]], [onnx::Concat_212 -> (1)[INT32]], [onnx::Concat_864 -> (1)[INT32]], [onnx::Concat_865 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_864 for ONNX node: onnx::Concat_864 [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_865 for ONNX node: onnx::Concat_865 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Concat_96 for ONNX node: Concat_96 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Reshape_215 for ONNX tensor: onnx::Reshape_215 [05/12/2023-15:13:48] [V] [TRT] Concat_96 [Concat] outputs: [onnx::Reshape_215 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Reshape_97 [Reshape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: mixed_query_layer [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Reshape_215 [05/12/2023-15:13:48] [V] [TRT] Reshape_97 [Reshape] inputs: [mixed_query_layer -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_215 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Reshape_97 for ONNX node: Reshape_97 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Transpose_216 for ONNX tensor: onnx::Transpose_216 [05/12/2023-15:13:48] [V] [TRT] Reshape_97 [Reshape] outputs: [onnx::Transpose_216 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Transpose_98 [Transpose] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Transpose_216 [05/12/2023-15:13:48] [V] [TRT] Transpose_98 [Transpose] inputs: [onnx::Transpose_216 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Transpose_98 for ONNX node: Transpose_98 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::MatMul_217 for ONNX tensor: onnx::MatMul_217 [05/12/2023-15:13:48] [V] [TRT] Transpose_98 [Transpose] outputs: [onnx::MatMul_217 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Transpose_99 [Transpose] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Transpose_184 [05/12/2023-15:13:48] [V] [TRT] Transpose_99 [Transpose] inputs: [onnx::Transpose_184 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Transpose_99 for ONNX node: Transpose_99 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::MatMul_218 for ONNX tensor: onnx::MatMul_218 [05/12/2023-15:13:48] [V] [TRT] Transpose_99 [Transpose] outputs: [onnx::MatMul_218 -> (-1, 12, 32, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_100 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_217 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_218 [05/12/2023-15:13:48] [V] [TRT] MatMul_100 [MatMul] inputs: [onnx::MatMul_217 -> (-1, 12, -1, 32)[FLOAT]], [onnx::MatMul_218 -> (-1, 12, 32, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_100 for ONNX node: MatMul_100 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Div_219 for ONNX tensor: onnx::Div_219 [05/12/2023-15:13:48] [V] [TRT] MatMul_100 [MatMul] outputs: [onnx::Div_219 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_101 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_101 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_101 [Constant] outputs: [onnx::Div_220 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Div_102 [Div] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Div_219 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Div_220 [05/12/2023-15:13:48] [V] [TRT] Div_102 [Div] inputs: [onnx::Div_219 -> (-1, 12, -1, -1)[FLOAT]], [onnx::Div_220 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Div_220 for ONNX node: onnx::Div_220 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Div_102 for ONNX node: Div_102 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_221 for ONNX tensor: onnx::Add_221 [05/12/2023-15:13:48] [V] [TRT] Div_102 [Div] outputs: [onnx::Add_221 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_103 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_221 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_138 [05/12/2023-15:13:48] [V] [TRT] Add_103 [Add] inputs: [onnx::Add_221 -> (-1, 12, -1, -1)[FLOAT]], [onnx::Add_138 -> (-1, 1, 1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_103 for ONNX node: Add_103 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: input.12 for ONNX tensor: input.12 [05/12/2023-15:13:48] [V] [TRT] Add_103 [Add] outputs: [input.12 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Softmax_104 [Softmax] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.12 [05/12/2023-15:13:48] [V] [TRT] Softmax_104 [Softmax] inputs: [input.12 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Softmax_104 for ONNX node: Softmax_104 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: input.16 for ONNX tensor: input.16 [05/12/2023-15:13:48] [V] [TRT] Softmax_104 [Softmax] outputs: [input.16 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_105 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.16 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_202 [05/12/2023-15:13:48] [V] [TRT] MatMul_105 [MatMul] inputs: [input.16 -> (-1, 12, -1, -1)[FLOAT]], [onnx::MatMul_202 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_105 for ONNX node: MatMul_105 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Transpose_224 for ONNX tensor: onnx::Transpose_224 [05/12/2023-15:13:48] [V] [TRT] MatMul_105 [MatMul] outputs: [onnx::Transpose_224 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Transpose_106 [Transpose] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Transpose_224 [05/12/2023-15:13:48] [V] [TRT] Transpose_106 [Transpose] inputs: [onnx::Transpose_224 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Transpose_106 for ONNX node: Transpose_106 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Shape_225 for ONNX tensor: onnx::Shape_225 [05/12/2023-15:13:48] [V] [TRT] Transpose_106 [Transpose] outputs: [onnx::Shape_225 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_107 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_225 [05/12/2023-15:13:48] [V] [TRT] Shape_107 [Shape] inputs: [onnx::Shape_225 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_107 for ONNX node: Shape_107 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_226 for ONNX tensor: onnx::Gather_226 [05/12/2023-15:13:48] [V] [TRT] Shape_107 [Shape] outputs: [onnx::Gather_226 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_108 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_108 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_108 [Constant] outputs: [onnx::Gather_227 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_109 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_226 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_227 [05/12/2023-15:13:48] [V] [TRT] Gather_109 [Gather] inputs: [onnx::Gather_226 -> (4)[INT32]], [onnx::Gather_227 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_227 for ONNX node: onnx::Gather_227 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_109 for ONNX node: Gather_109 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_228 for ONNX tensor: onnx::Unsqueeze_228 [05/12/2023-15:13:48] [V] [TRT] Gather_109 [Gather] outputs: [onnx::Unsqueeze_228 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_110 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_225 [05/12/2023-15:13:48] [V] [TRT] Shape_110 [Shape] inputs: [onnx::Shape_225 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_110 for ONNX node: Shape_110 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_229 for ONNX tensor: onnx::Gather_229 [05/12/2023-15:13:48] [V] [TRT] Shape_110 [Shape] outputs: [onnx::Gather_229 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_111 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_111 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_111 [Constant] outputs: [onnx::Gather_230 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_112 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_229 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_230 [05/12/2023-15:13:48] [V] [TRT] Gather_112 [Gather] inputs: [onnx::Gather_229 -> (4)[INT32]], [onnx::Gather_230 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_230 for ONNX node: onnx::Gather_230 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_112 for ONNX node: Gather_112 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_231 for ONNX tensor: onnx::Unsqueeze_231 [05/12/2023-15:13:48] [V] [TRT] Gather_112 [Gather] outputs: [onnx::Unsqueeze_231 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_113 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_228 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_113 [Unsqueeze] inputs: [onnx::Unsqueeze_228 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_113 for ONNX node: Unsqueeze_113 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_233 for ONNX tensor: onnx::Concat_233 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_113 [Unsqueeze] outputs: [onnx::Concat_233 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_114 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_231 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_114 [Unsqueeze] inputs: [onnx::Unsqueeze_231 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_114 for ONNX node: Unsqueeze_114 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_234 for ONNX tensor: onnx::Concat_234 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_114 [Unsqueeze] outputs: [onnx::Concat_234 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_115 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_115 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_115 [Constant] outputs: [onnx::Concat_866 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Concat_116 [Concat] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_233 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_234 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_866 [05/12/2023-15:13:48] [V] [TRT] Concat_116 [Concat] inputs: [onnx::Concat_233 -> (1)[INT32]], [onnx::Concat_234 -> (1)[INT32]], [onnx::Concat_866 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_866 for ONNX node: onnx::Concat_866 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Concat_116 for ONNX node: Concat_116 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Reshape_236 for ONNX tensor: onnx::Reshape_236 [05/12/2023-15:13:48] [V] [TRT] Concat_116 [Concat] outputs: [onnx::Reshape_236 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Reshape_117 [Reshape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_225 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Reshape_236 [05/12/2023-15:13:48] [V] [TRT] Reshape_117 [Reshape] inputs: [onnx::Shape_225 -> (-1, -1, 12, 32)[FLOAT]], [onnx::Reshape_236 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Reshape_117 for ONNX node: Reshape_117 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::MatMul_237 for ONNX tensor: onnx::MatMul_237 [05/12/2023-15:13:48] [V] [TRT] Reshape_117 [Reshape] outputs: [onnx::MatMul_237 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_118 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_237 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_867 [05/12/2023-15:13:48] [V] [TRT] MatMul_118 [MatMul] inputs: [onnx::MatMul_237 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_867 -> (384, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::MatMul_867 for ONNX node: onnx::MatMul_867 [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_118 for ONNX node: MatMul_118 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_239 for ONNX tensor: onnx::Add_239 [05/12/2023-15:13:48] [V] [TRT] MatMul_118 [MatMul] outputs: [onnx::Add_239 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_119 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.0.attention.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_239 [05/12/2023-15:13:48] [V] [TRT] Add_119 [Add] inputs: [encoder.layer.0.attention.output.dense.bias -> (384)[FLOAT]], [onnx::Add_239 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.0.attention.output.dense.bias for ONNX node: encoder.layer.0.attention.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_119 for ONNX node: Add_119 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: input.20 for ONNX tensor: input.20 [05/12/2023-15:13:48] [V] [TRT] Add_119 [Add] outputs: [input.20 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_120 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.20 [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.8 [05/12/2023-15:13:48] [V] [TRT] Add_120 [Add] inputs: [input.20 -> (-1, -1, 384)[FLOAT]], [input.8 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_120 for ONNX node: Add_120 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: input.24 for ONNX tensor: input.24 [05/12/2023-15:13:48] [V] [TRT] Add_120 [Add] outputs: [input.24 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: ReduceMean_121 [ReduceMean] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.24 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_121 [ReduceMean] inputs: [input.24 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: ReduceMean_121 for ONNX node: ReduceMean_121 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Sub_242 for ONNX tensor: onnx::Sub_242 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_121 [ReduceMean] outputs: [onnx::Sub_242 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Sub_122 [Sub] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.24 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Sub_242 [05/12/2023-15:13:48] [V] [TRT] Sub_122 [Sub] inputs: [input.24 -> (-1, -1, 384)[FLOAT]], [onnx::Sub_242 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Sub_122 for ONNX node: Sub_122 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Pow_243 for ONNX tensor: onnx::Pow_243 [05/12/2023-15:13:48] [V] [TRT] Sub_122 [Sub] outputs: [onnx::Pow_243 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_123 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_123 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_123 [Constant] outputs: [onnx::Pow_244 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Pow_124 [Pow] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Pow_243 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Pow_244 [05/12/2023-15:13:48] [V] [TRT] Pow_124 [Pow] inputs: [onnx::Pow_243 -> (-1, -1, 384)[FLOAT]], [onnx::Pow_244 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Pow_244 for ONNX node: onnx::Pow_244 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Pow_124 for ONNX node: Pow_124 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::ReduceMean_245 for ONNX tensor: onnx::ReduceMean_245 [05/12/2023-15:13:48] [V] [TRT] Pow_124 [Pow] outputs: [onnx::ReduceMean_245 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: ReduceMean_125 [ReduceMean] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::ReduceMean_245 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_125 [ReduceMean] inputs: [onnx::ReduceMean_245 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: ReduceMean_125 for ONNX node: ReduceMean_125 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_246 for ONNX tensor: onnx::Add_246 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_125 [ReduceMean] outputs: [onnx::Add_246 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_126 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_126 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_126 [Constant] outputs: [onnx::Add_247 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_127 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_246 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_247 [05/12/2023-15:13:48] [V] [TRT] Add_127 [Add] inputs: [onnx::Add_246 -> (-1, -1, 1)[FLOAT]], [onnx::Add_247 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Add_247 for ONNX node: onnx::Add_247 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_127 for ONNX node: Add_127 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Sqrt_248 for ONNX tensor: onnx::Sqrt_248 [05/12/2023-15:13:48] [V] [TRT] Add_127 [Add] outputs: [onnx::Sqrt_248 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Sqrt_128 [Sqrt] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Sqrt_248 [05/12/2023-15:13:48] [V] [TRT] Sqrt_128 [Sqrt] inputs: [onnx::Sqrt_248 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Sqrt_128 for ONNX node: Sqrt_128 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Div_249 for ONNX tensor: onnx::Div_249 [05/12/2023-15:13:48] [V] [TRT] Sqrt_128 [Sqrt] outputs: [onnx::Div_249 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Div_129 [Div] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Pow_243 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Div_249 [05/12/2023-15:13:48] [V] [TRT] Div_129 [Div] inputs: [onnx::Pow_243 -> (-1, -1, 384)[FLOAT]], [onnx::Div_249 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Div_129 for ONNX node: Div_129 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Mul_250 for ONNX tensor: onnx::Mul_250 [05/12/2023-15:13:48] [V] [TRT] Div_129 [Div] outputs: [onnx::Mul_250 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Mul_130 [Mul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Mul_250 [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.0.attention.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Mul_130 [Mul] inputs: [onnx::Mul_250 -> (-1, -1, 384)[FLOAT]], [encoder.layer.0.attention.output.LayerNorm.weight -> (384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.0.attention.output.LayerNorm.weight for ONNX node: encoder.layer.0.attention.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Registering layer: Mul_130 for ONNX node: Mul_130 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_251 for ONNX tensor: onnx::Add_251 [05/12/2023-15:13:48] [V] [TRT] Mul_130 [Mul] outputs: [onnx::Add_251 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_131 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_251 [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.0.attention.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Add_131 [Add] inputs: [onnx::Add_251 -> (-1, -1, 384)[FLOAT]], [encoder.layer.0.attention.output.LayerNorm.bias -> (384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.0.attention.output.LayerNorm.bias for ONNX node: encoder.layer.0.attention.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_131 for ONNX node: Add_131 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: attention_output for ONNX tensor: attention_output [05/12/2023-15:13:48] [V] [TRT] Add_131 [Add] outputs: [attention_output -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_132 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: attention_output [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_868 [05/12/2023-15:13:48] [V] [TRT] MatMul_132 [MatMul] inputs: [attention_output -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_868 -> (384, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::MatMul_868 for ONNX node: onnx::MatMul_868 [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_132 for ONNX node: MatMul_132 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_254 for ONNX tensor: onnx::Add_254 [05/12/2023-15:13:48] [V] [TRT] MatMul_132 [MatMul] outputs: [onnx::Add_254 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_133 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.0.intermediate.dense.bias [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_254 [05/12/2023-15:13:48] [V] [TRT] Add_133 [Add] inputs: [encoder.layer.0.intermediate.dense.bias -> (1536)[FLOAT]], [onnx::Add_254 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.0.intermediate.dense.bias for ONNX node: encoder.layer.0.intermediate.dense.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_133 for ONNX node: Add_133 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Div_255 for ONNX tensor: onnx::Div_255 [05/12/2023-15:13:48] [V] [TRT] Add_133 [Add] outputs: [onnx::Div_255 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_134 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_134 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_134 [Constant] outputs: [onnx::Div_256 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Div_135 [Div] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Div_255 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Div_256 [05/12/2023-15:13:48] [V] [TRT] Div_135 [Div] inputs: [onnx::Div_255 -> (-1, -1, 1536)[FLOAT]], [onnx::Div_256 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Div_256 for ONNX node: onnx::Div_256 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Div_135 for ONNX node: Div_135 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Erf_257 for ONNX tensor: onnx::Erf_257 [05/12/2023-15:13:48] [V] [TRT] Div_135 [Div] outputs: [onnx::Erf_257 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Erf_136 [Erf] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Erf_257 [05/12/2023-15:13:48] [V] [TRT] Erf_136 [Erf] inputs: [onnx::Erf_257 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Erf_136 for ONNX node: Erf_136 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_258 for ONNX tensor: onnx::Add_258 [05/12/2023-15:13:48] [V] [TRT] Erf_136 [Erf] outputs: [onnx::Add_258 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_137 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_137 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_137 [Constant] outputs: [onnx::Add_259 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_138 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_258 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_259 [05/12/2023-15:13:48] [V] [TRT] Add_138 [Add] inputs: [onnx::Add_258 -> (-1, -1, 1536)[FLOAT]], [onnx::Add_259 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Add_259 for ONNX node: onnx::Add_259 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_138 for ONNX node: Add_138 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Mul_260 for ONNX tensor: onnx::Mul_260 [05/12/2023-15:13:48] [V] [TRT] Add_138 [Add] outputs: [onnx::Mul_260 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Mul_139 [Mul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Div_255 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Mul_260 [05/12/2023-15:13:48] [V] [TRT] Mul_139 [Mul] inputs: [onnx::Div_255 -> (-1, -1, 1536)[FLOAT]], [onnx::Mul_260 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Mul_139 for ONNX node: Mul_139 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Mul_261 for ONNX tensor: onnx::Mul_261 [05/12/2023-15:13:48] [V] [TRT] Mul_139 [Mul] outputs: [onnx::Mul_261 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_140 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_140 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_140 [Constant] outputs: [onnx::Mul_262 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Mul_141 [Mul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Mul_261 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Mul_262 [05/12/2023-15:13:48] [V] [TRT] Mul_141 [Mul] inputs: [onnx::Mul_261 -> (-1, -1, 1536)[FLOAT]], [onnx::Mul_262 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Mul_262 for ONNX node: onnx::Mul_262 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Mul_141 for ONNX node: Mul_141 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::MatMul_263 for ONNX tensor: onnx::MatMul_263 [05/12/2023-15:13:48] [V] [TRT] Mul_141 [Mul] outputs: [onnx::MatMul_263 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_142 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_263 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_869 [05/12/2023-15:13:48] [V] [TRT] MatMul_142 [MatMul] inputs: [onnx::MatMul_263 -> (-1, -1, 1536)[FLOAT]], [onnx::MatMul_869 -> (1536, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::MatMul_869 for ONNX node: onnx::MatMul_869 [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_142 for ONNX node: MatMul_142 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_265 for ONNX tensor: onnx::Add_265 [05/12/2023-15:13:48] [V] [TRT] MatMul_142 [MatMul] outputs: [onnx::Add_265 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_143 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.0.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_265 [05/12/2023-15:13:48] [V] [TRT] Add_143 [Add] inputs: [encoder.layer.0.output.dense.bias -> (384)[FLOAT]], [onnx::Add_265 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.0.output.dense.bias for ONNX node: encoder.layer.0.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_143 for ONNX node: Add_143 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: input.28 for ONNX tensor: input.28 [05/12/2023-15:13:48] [V] [TRT] Add_143 [Add] outputs: [input.28 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_144 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.28 [05/12/2023-15:13:48] [V] [TRT] Searching for input: attention_output [05/12/2023-15:13:48] [V] [TRT] Add_144 [Add] inputs: [input.28 -> (-1, -1, 384)[FLOAT]], [attention_output -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_144 for ONNX node: Add_144 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: input.32 for ONNX tensor: input.32 [05/12/2023-15:13:48] [V] [TRT] Add_144 [Add] outputs: [input.32 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: ReduceMean_145 [ReduceMean] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.32 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_145 [ReduceMean] inputs: [input.32 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: ReduceMean_145 for ONNX node: ReduceMean_145 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Sub_268 for ONNX tensor: onnx::Sub_268 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_145 [ReduceMean] outputs: [onnx::Sub_268 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Sub_146 [Sub] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.32 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Sub_268 [05/12/2023-15:13:48] [V] [TRT] Sub_146 [Sub] inputs: [input.32 -> (-1, -1, 384)[FLOAT]], [onnx::Sub_268 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Sub_146 for ONNX node: Sub_146 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Pow_269 for ONNX tensor: onnx::Pow_269 [05/12/2023-15:13:48] [V] [TRT] Sub_146 [Sub] outputs: [onnx::Pow_269 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_147 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_147 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_147 [Constant] outputs: [onnx::Pow_270 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Pow_148 [Pow] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Pow_269 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Pow_270 [05/12/2023-15:13:48] [V] [TRT] Pow_148 [Pow] inputs: [onnx::Pow_269 -> (-1, -1, 384)[FLOAT]], [onnx::Pow_270 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Pow_270 for ONNX node: onnx::Pow_270 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Pow_148 for ONNX node: Pow_148 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::ReduceMean_271 for ONNX tensor: onnx::ReduceMean_271 [05/12/2023-15:13:48] [V] [TRT] Pow_148 [Pow] outputs: [onnx::ReduceMean_271 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: ReduceMean_149 [ReduceMean] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::ReduceMean_271 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_149 [ReduceMean] inputs: [onnx::ReduceMean_271 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: ReduceMean_149 for ONNX node: ReduceMean_149 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_272 for ONNX tensor: onnx::Add_272 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_149 [ReduceMean] outputs: [onnx::Add_272 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_150 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_150 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_150 [Constant] outputs: [onnx::Add_273 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_151 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_272 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_273 [05/12/2023-15:13:48] [V] [TRT] Add_151 [Add] inputs: [onnx::Add_272 -> (-1, -1, 1)[FLOAT]], [onnx::Add_273 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Add_273 for ONNX node: onnx::Add_273 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_151 for ONNX node: Add_151 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Sqrt_274 for ONNX tensor: onnx::Sqrt_274 [05/12/2023-15:13:48] [V] [TRT] Add_151 [Add] outputs: [onnx::Sqrt_274 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Sqrt_152 [Sqrt] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Sqrt_274 [05/12/2023-15:13:48] [V] [TRT] Sqrt_152 [Sqrt] inputs: [onnx::Sqrt_274 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Sqrt_152 for ONNX node: Sqrt_152 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Div_275 for ONNX tensor: onnx::Div_275 [05/12/2023-15:13:48] [V] [TRT] Sqrt_152 [Sqrt] outputs: [onnx::Div_275 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Div_153 [Div] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Pow_269 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Div_275 [05/12/2023-15:13:48] [V] [TRT] Div_153 [Div] inputs: [onnx::Pow_269 -> (-1, -1, 384)[FLOAT]], [onnx::Div_275 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Div_153 for ONNX node: Div_153 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Mul_276 for ONNX tensor: onnx::Mul_276 [05/12/2023-15:13:48] [V] [TRT] Div_153 [Div] outputs: [onnx::Mul_276 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Mul_154 [Mul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Mul_276 [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.0.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Mul_154 [Mul] inputs: [onnx::Mul_276 -> (-1, -1, 384)[FLOAT]], [encoder.layer.0.output.LayerNorm.weight -> (384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.0.output.LayerNorm.weight for ONNX node: encoder.layer.0.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Registering layer: Mul_154 for ONNX node: Mul_154 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_277 for ONNX tensor: onnx::Add_277 [05/12/2023-15:13:48] [V] [TRT] Mul_154 [Mul] outputs: [onnx::Add_277 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_155 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_277 [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.0.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Add_155 [Add] inputs: [onnx::Add_277 -> (-1, -1, 384)[FLOAT]], [encoder.layer.0.output.LayerNorm.bias -> (384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.0.output.LayerNorm.bias for ONNX node: encoder.layer.0.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_155 for ONNX node: Add_155 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::MatMul_278 for ONNX tensor: onnx::MatMul_278 [05/12/2023-15:13:48] [V] [TRT] Add_155 [Add] outputs: [onnx::MatMul_278 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_156 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_278 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_870 [05/12/2023-15:13:48] [V] [TRT] MatMul_156 [MatMul] inputs: [onnx::MatMul_278 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_870 -> (384, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::MatMul_870 for ONNX node: onnx::MatMul_870 [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_156 for ONNX node: MatMul_156 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_280 for ONNX tensor: onnx::Add_280 [05/12/2023-15:13:48] [V] [TRT] MatMul_156 [MatMul] outputs: [onnx::Add_280 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_157 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.1.attention.self.query.bias [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_280 [05/12/2023-15:13:48] [V] [TRT] Add_157 [Add] inputs: [encoder.layer.1.attention.self.query.bias -> (384)[FLOAT]], [onnx::Add_280 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.1.attention.self.query.bias for ONNX node: encoder.layer.1.attention.self.query.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_157 for ONNX node: Add_157 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: mixed_query_layer.3 for ONNX tensor: mixed_query_layer.3 [05/12/2023-15:13:48] [V] [TRT] Add_157 [Add] outputs: [mixed_query_layer.3 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_158 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_278 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_871 [05/12/2023-15:13:48] [V] [TRT] MatMul_158 [MatMul] inputs: [onnx::MatMul_278 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_871 -> (384, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::MatMul_871 for ONNX node: onnx::MatMul_871 [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_158 for ONNX node: MatMul_158 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_283 for ONNX tensor: onnx::Add_283 [05/12/2023-15:13:48] [V] [TRT] MatMul_158 [MatMul] outputs: [onnx::Add_283 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_159 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.1.attention.self.key.bias [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_283 [05/12/2023-15:13:48] [V] [TRT] Add_159 [Add] inputs: [encoder.layer.1.attention.self.key.bias -> (384)[FLOAT]], [onnx::Add_283 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.1.attention.self.key.bias for ONNX node: encoder.layer.1.attention.self.key.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_159 for ONNX node: Add_159 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Shape_284 for ONNX tensor: onnx::Shape_284 [05/12/2023-15:13:48] [V] [TRT] Add_159 [Add] outputs: [onnx::Shape_284 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_160 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_284 [05/12/2023-15:13:48] [V] [TRT] Shape_160 [Shape] inputs: [onnx::Shape_284 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_160 for ONNX node: Shape_160 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_285 for ONNX tensor: onnx::Gather_285 [05/12/2023-15:13:48] [V] [TRT] Shape_160 [Shape] outputs: [onnx::Gather_285 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_161 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_161 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_161 [Constant] outputs: [onnx::Gather_286 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_162 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_285 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_286 [05/12/2023-15:13:48] [V] [TRT] Gather_162 [Gather] inputs: [onnx::Gather_285 -> (3)[INT32]], [onnx::Gather_286 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_286 for ONNX node: onnx::Gather_286 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_162 for ONNX node: Gather_162 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_287 for ONNX tensor: onnx::Unsqueeze_287 [05/12/2023-15:13:48] [V] [TRT] Gather_162 [Gather] outputs: [onnx::Unsqueeze_287 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_163 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_284 [05/12/2023-15:13:48] [V] [TRT] Shape_163 [Shape] inputs: [onnx::Shape_284 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_163 for ONNX node: Shape_163 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_288 for ONNX tensor: onnx::Gather_288 [05/12/2023-15:13:48] [V] [TRT] Shape_163 [Shape] outputs: [onnx::Gather_288 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_164 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_164 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_164 [Constant] outputs: [onnx::Gather_289 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_165 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_288 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_289 [05/12/2023-15:13:48] [V] [TRT] Gather_165 [Gather] inputs: [onnx::Gather_288 -> (3)[INT32]], [onnx::Gather_289 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_289 for ONNX node: onnx::Gather_289 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_165 for ONNX node: Gather_165 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_290 for ONNX tensor: onnx::Unsqueeze_290 [05/12/2023-15:13:48] [V] [TRT] Gather_165 [Gather] outputs: [onnx::Unsqueeze_290 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_166 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_287 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_166 [Unsqueeze] inputs: [onnx::Unsqueeze_287 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_166 for ONNX node: Unsqueeze_166 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_293 for ONNX tensor: onnx::Concat_293 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_166 [Unsqueeze] outputs: [onnx::Concat_293 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_167 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_290 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_167 [Unsqueeze] inputs: [onnx::Unsqueeze_290 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_167 for ONNX node: Unsqueeze_167 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_294 for ONNX tensor: onnx::Concat_294 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_167 [Unsqueeze] outputs: [onnx::Concat_294 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_168 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_168 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_168 [Constant] outputs: [onnx::Concat_872 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_169 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_169 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_169 [Constant] outputs: [onnx::Concat_873 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Concat_170 [Concat] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_293 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_294 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_872 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_873 [05/12/2023-15:13:48] [V] [TRT] Concat_170 [Concat] inputs: [onnx::Concat_293 -> (1)[INT32]], [onnx::Concat_294 -> (1)[INT32]], [onnx::Concat_872 -> (1)[INT32]], [onnx::Concat_873 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_872 for ONNX node: onnx::Concat_872 [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_873 for ONNX node: onnx::Concat_873 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Concat_170 for ONNX node: Concat_170 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Reshape_297 for ONNX tensor: onnx::Reshape_297 [05/12/2023-15:13:48] [V] [TRT] Concat_170 [Concat] outputs: [onnx::Reshape_297 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Reshape_171 [Reshape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_284 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Reshape_297 [05/12/2023-15:13:48] [V] [TRT] Reshape_171 [Reshape] inputs: [onnx::Shape_284 -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_297 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Reshape_171 for ONNX node: Reshape_171 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Transpose_298 for ONNX tensor: onnx::Transpose_298 [05/12/2023-15:13:48] [V] [TRT] Reshape_171 [Reshape] outputs: [onnx::Transpose_298 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_172 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_278 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_874 [05/12/2023-15:13:48] [V] [TRT] MatMul_172 [MatMul] inputs: [onnx::MatMul_278 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_874 -> (384, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::MatMul_874 for ONNX node: onnx::MatMul_874 [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_172 for ONNX node: MatMul_172 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_300 for ONNX tensor: onnx::Add_300 [05/12/2023-15:13:48] [V] [TRT] MatMul_172 [MatMul] outputs: [onnx::Add_300 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_173 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.1.attention.self.value.bias [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_300 [05/12/2023-15:13:48] [V] [TRT] Add_173 [Add] inputs: [encoder.layer.1.attention.self.value.bias -> (384)[FLOAT]], [onnx::Add_300 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.1.attention.self.value.bias for ONNX node: encoder.layer.1.attention.self.value.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_173 for ONNX node: Add_173 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Shape_301 for ONNX tensor: onnx::Shape_301 [05/12/2023-15:13:48] [V] [TRT] Add_173 [Add] outputs: [onnx::Shape_301 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_174 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_301 [05/12/2023-15:13:48] [V] [TRT] Shape_174 [Shape] inputs: [onnx::Shape_301 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_174 for ONNX node: Shape_174 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_302 for ONNX tensor: onnx::Gather_302 [05/12/2023-15:13:48] [V] [TRT] Shape_174 [Shape] outputs: [onnx::Gather_302 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_175 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_175 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_175 [Constant] outputs: [onnx::Gather_303 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_176 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_302 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_303 [05/12/2023-15:13:48] [V] [TRT] Gather_176 [Gather] inputs: [onnx::Gather_302 -> (3)[INT32]], [onnx::Gather_303 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_303 for ONNX node: onnx::Gather_303 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_176 for ONNX node: Gather_176 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_304 for ONNX tensor: onnx::Unsqueeze_304 [05/12/2023-15:13:48] [V] [TRT] Gather_176 [Gather] outputs: [onnx::Unsqueeze_304 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_177 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_301 [05/12/2023-15:13:48] [V] [TRT] Shape_177 [Shape] inputs: [onnx::Shape_301 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_177 for ONNX node: Shape_177 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_305 for ONNX tensor: onnx::Gather_305 [05/12/2023-15:13:48] [V] [TRT] Shape_177 [Shape] outputs: [onnx::Gather_305 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_178 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_178 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_178 [Constant] outputs: [onnx::Gather_306 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_179 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_305 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_306 [05/12/2023-15:13:48] [V] [TRT] Gather_179 [Gather] inputs: [onnx::Gather_305 -> (3)[INT32]], [onnx::Gather_306 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_306 for ONNX node: onnx::Gather_306 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_179 for ONNX node: Gather_179 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_307 for ONNX tensor: onnx::Unsqueeze_307 [05/12/2023-15:13:48] [V] [TRT] Gather_179 [Gather] outputs: [onnx::Unsqueeze_307 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_180 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_304 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_180 [Unsqueeze] inputs: [onnx::Unsqueeze_304 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_180 for ONNX node: Unsqueeze_180 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_310 for ONNX tensor: onnx::Concat_310 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_180 [Unsqueeze] outputs: [onnx::Concat_310 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_181 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_307 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_181 [Unsqueeze] inputs: [onnx::Unsqueeze_307 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_181 for ONNX node: Unsqueeze_181 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_311 for ONNX tensor: onnx::Concat_311 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_181 [Unsqueeze] outputs: [onnx::Concat_311 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_182 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_182 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_182 [Constant] outputs: [onnx::Concat_875 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_183 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_183 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_183 [Constant] outputs: [onnx::Concat_876 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Concat_184 [Concat] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_310 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_311 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_875 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_876 [05/12/2023-15:13:48] [V] [TRT] Concat_184 [Concat] inputs: [onnx::Concat_310 -> (1)[INT32]], [onnx::Concat_311 -> (1)[INT32]], [onnx::Concat_875 -> (1)[INT32]], [onnx::Concat_876 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_875 for ONNX node: onnx::Concat_875 [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_876 for ONNX node: onnx::Concat_876 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Concat_184 for ONNX node: Concat_184 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Reshape_314 for ONNX tensor: onnx::Reshape_314 [05/12/2023-15:13:48] [V] [TRT] Concat_184 [Concat] outputs: [onnx::Reshape_314 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Reshape_185 [Reshape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_301 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Reshape_314 [05/12/2023-15:13:48] [V] [TRT] Reshape_185 [Reshape] inputs: [onnx::Shape_301 -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_314 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Reshape_185 for ONNX node: Reshape_185 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Transpose_315 for ONNX tensor: onnx::Transpose_315 [05/12/2023-15:13:48] [V] [TRT] Reshape_185 [Reshape] outputs: [onnx::Transpose_315 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Transpose_186 [Transpose] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Transpose_315 [05/12/2023-15:13:48] [V] [TRT] Transpose_186 [Transpose] inputs: [onnx::Transpose_315 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Transpose_186 for ONNX node: Transpose_186 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::MatMul_316 for ONNX tensor: onnx::MatMul_316 [05/12/2023-15:13:48] [V] [TRT] Transpose_186 [Transpose] outputs: [onnx::MatMul_316 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_187 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: mixed_query_layer.3 [05/12/2023-15:13:48] [V] [TRT] Shape_187 [Shape] inputs: [mixed_query_layer.3 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_187 for ONNX node: Shape_187 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_317 for ONNX tensor: onnx::Gather_317 [05/12/2023-15:13:48] [V] [TRT] Shape_187 [Shape] outputs: [onnx::Gather_317 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_188 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_188 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_188 [Constant] outputs: [onnx::Gather_318 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_189 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_317 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_318 [05/12/2023-15:13:48] [V] [TRT] Gather_189 [Gather] inputs: [onnx::Gather_317 -> (3)[INT32]], [onnx::Gather_318 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_318 for ONNX node: onnx::Gather_318 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_189 for ONNX node: Gather_189 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_319 for ONNX tensor: onnx::Unsqueeze_319 [05/12/2023-15:13:48] [V] [TRT] Gather_189 [Gather] outputs: [onnx::Unsqueeze_319 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_190 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: mixed_query_layer.3 [05/12/2023-15:13:48] [V] [TRT] Shape_190 [Shape] inputs: [mixed_query_layer.3 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_190 for ONNX node: Shape_190 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_320 for ONNX tensor: onnx::Gather_320 [05/12/2023-15:13:48] [V] [TRT] Shape_190 [Shape] outputs: [onnx::Gather_320 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_191 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_191 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_191 [Constant] outputs: [onnx::Gather_321 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_192 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_320 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_321 [05/12/2023-15:13:48] [V] [TRT] Gather_192 [Gather] inputs: [onnx::Gather_320 -> (3)[INT32]], [onnx::Gather_321 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_321 for ONNX node: onnx::Gather_321 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_192 for ONNX node: Gather_192 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_322 for ONNX tensor: onnx::Unsqueeze_322 [05/12/2023-15:13:48] [V] [TRT] Gather_192 [Gather] outputs: [onnx::Unsqueeze_322 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_193 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_319 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_193 [Unsqueeze] inputs: [onnx::Unsqueeze_319 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_193 for ONNX node: Unsqueeze_193 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_325 for ONNX tensor: onnx::Concat_325 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_193 [Unsqueeze] outputs: [onnx::Concat_325 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_194 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_322 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_194 [Unsqueeze] inputs: [onnx::Unsqueeze_322 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_194 for ONNX node: Unsqueeze_194 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_326 for ONNX tensor: onnx::Concat_326 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_194 [Unsqueeze] outputs: [onnx::Concat_326 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_195 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_195 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_195 [Constant] outputs: [onnx::Concat_877 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_196 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_196 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_196 [Constant] outputs: [onnx::Concat_878 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Concat_197 [Concat] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_325 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_326 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_877 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_878 [05/12/2023-15:13:48] [V] [TRT] Concat_197 [Concat] inputs: [onnx::Concat_325 -> (1)[INT32]], [onnx::Concat_326 -> (1)[INT32]], [onnx::Concat_877 -> (1)[INT32]], [onnx::Concat_878 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_877 for ONNX node: onnx::Concat_877 [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_878 for ONNX node: onnx::Concat_878 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Concat_197 for ONNX node: Concat_197 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Reshape_329 for ONNX tensor: onnx::Reshape_329 [05/12/2023-15:13:48] [V] [TRT] Concat_197 [Concat] outputs: [onnx::Reshape_329 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Reshape_198 [Reshape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: mixed_query_layer.3 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Reshape_329 [05/12/2023-15:13:48] [V] [TRT] Reshape_198 [Reshape] inputs: [mixed_query_layer.3 -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_329 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Reshape_198 for ONNX node: Reshape_198 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Transpose_330 for ONNX tensor: onnx::Transpose_330 [05/12/2023-15:13:48] [V] [TRT] Reshape_198 [Reshape] outputs: [onnx::Transpose_330 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Transpose_199 [Transpose] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Transpose_330 [05/12/2023-15:13:48] [V] [TRT] Transpose_199 [Transpose] inputs: [onnx::Transpose_330 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Transpose_199 for ONNX node: Transpose_199 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::MatMul_331 for ONNX tensor: onnx::MatMul_331 [05/12/2023-15:13:48] [V] [TRT] Transpose_199 [Transpose] outputs: [onnx::MatMul_331 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Transpose_200 [Transpose] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Transpose_298 [05/12/2023-15:13:48] [V] [TRT] Transpose_200 [Transpose] inputs: [onnx::Transpose_298 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Transpose_200 for ONNX node: Transpose_200 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::MatMul_332 for ONNX tensor: onnx::MatMul_332 [05/12/2023-15:13:48] [V] [TRT] Transpose_200 [Transpose] outputs: [onnx::MatMul_332 -> (-1, 12, 32, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_201 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_331 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_332 [05/12/2023-15:13:48] [V] [TRT] MatMul_201 [MatMul] inputs: [onnx::MatMul_331 -> (-1, 12, -1, 32)[FLOAT]], [onnx::MatMul_332 -> (-1, 12, 32, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_201 for ONNX node: MatMul_201 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Div_333 for ONNX tensor: onnx::Div_333 [05/12/2023-15:13:48] [V] [TRT] MatMul_201 [MatMul] outputs: [onnx::Div_333 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_202 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_202 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_202 [Constant] outputs: [onnx::Div_334 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Div_203 [Div] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Div_333 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Div_334 [05/12/2023-15:13:48] [V] [TRT] Div_203 [Div] inputs: [onnx::Div_333 -> (-1, 12, -1, -1)[FLOAT]], [onnx::Div_334 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Div_334 for ONNX node: onnx::Div_334 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Div_203 for ONNX node: Div_203 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_335 for ONNX tensor: onnx::Add_335 [05/12/2023-15:13:48] [V] [TRT] Div_203 [Div] outputs: [onnx::Add_335 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_204 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_335 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_138 [05/12/2023-15:13:48] [V] [TRT] Add_204 [Add] inputs: [onnx::Add_335 -> (-1, 12, -1, -1)[FLOAT]], [onnx::Add_138 -> (-1, 1, 1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_204 for ONNX node: Add_204 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: input.36 for ONNX tensor: input.36 [05/12/2023-15:13:48] [V] [TRT] Add_204 [Add] outputs: [input.36 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Softmax_205 [Softmax] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.36 [05/12/2023-15:13:48] [V] [TRT] Softmax_205 [Softmax] inputs: [input.36 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Softmax_205 for ONNX node: Softmax_205 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: input.40 for ONNX tensor: input.40 [05/12/2023-15:13:48] [V] [TRT] Softmax_205 [Softmax] outputs: [input.40 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_206 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.40 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_316 [05/12/2023-15:13:48] [V] [TRT] MatMul_206 [MatMul] inputs: [input.40 -> (-1, 12, -1, -1)[FLOAT]], [onnx::MatMul_316 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_206 for ONNX node: MatMul_206 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Transpose_338 for ONNX tensor: onnx::Transpose_338 [05/12/2023-15:13:48] [V] [TRT] MatMul_206 [MatMul] outputs: [onnx::Transpose_338 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Transpose_207 [Transpose] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Transpose_338 [05/12/2023-15:13:48] [V] [TRT] Transpose_207 [Transpose] inputs: [onnx::Transpose_338 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Transpose_207 for ONNX node: Transpose_207 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Shape_339 for ONNX tensor: onnx::Shape_339 [05/12/2023-15:13:48] [V] [TRT] Transpose_207 [Transpose] outputs: [onnx::Shape_339 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_208 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_339 [05/12/2023-15:13:48] [V] [TRT] Shape_208 [Shape] inputs: [onnx::Shape_339 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_208 for ONNX node: Shape_208 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_340 for ONNX tensor: onnx::Gather_340 [05/12/2023-15:13:48] [V] [TRT] Shape_208 [Shape] outputs: [onnx::Gather_340 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_209 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_209 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_209 [Constant] outputs: [onnx::Gather_341 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_210 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_340 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_341 [05/12/2023-15:13:48] [V] [TRT] Gather_210 [Gather] inputs: [onnx::Gather_340 -> (4)[INT32]], [onnx::Gather_341 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_341 for ONNX node: onnx::Gather_341 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_210 for ONNX node: Gather_210 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_342 for ONNX tensor: onnx::Unsqueeze_342 [05/12/2023-15:13:48] [V] [TRT] Gather_210 [Gather] outputs: [onnx::Unsqueeze_342 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_211 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_339 [05/12/2023-15:13:48] [V] [TRT] Shape_211 [Shape] inputs: [onnx::Shape_339 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_211 for ONNX node: Shape_211 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_343 for ONNX tensor: onnx::Gather_343 [05/12/2023-15:13:48] [V] [TRT] Shape_211 [Shape] outputs: [onnx::Gather_343 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_212 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_212 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_212 [Constant] outputs: [onnx::Gather_344 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_213 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_343 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_344 [05/12/2023-15:13:48] [V] [TRT] Gather_213 [Gather] inputs: [onnx::Gather_343 -> (4)[INT32]], [onnx::Gather_344 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_344 for ONNX node: onnx::Gather_344 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_213 for ONNX node: Gather_213 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_345 for ONNX tensor: onnx::Unsqueeze_345 [05/12/2023-15:13:48] [V] [TRT] Gather_213 [Gather] outputs: [onnx::Unsqueeze_345 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_214 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_342 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_214 [Unsqueeze] inputs: [onnx::Unsqueeze_342 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_214 for ONNX node: Unsqueeze_214 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_347 for ONNX tensor: onnx::Concat_347 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_214 [Unsqueeze] outputs: [onnx::Concat_347 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_215 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_345 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_215 [Unsqueeze] inputs: [onnx::Unsqueeze_345 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_215 for ONNX node: Unsqueeze_215 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_348 for ONNX tensor: onnx::Concat_348 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_215 [Unsqueeze] outputs: [onnx::Concat_348 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_216 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_216 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_216 [Constant] outputs: [onnx::Concat_879 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Concat_217 [Concat] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_347 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_348 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_879 [05/12/2023-15:13:48] [V] [TRT] Concat_217 [Concat] inputs: [onnx::Concat_347 -> (1)[INT32]], [onnx::Concat_348 -> (1)[INT32]], [onnx::Concat_879 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_879 for ONNX node: onnx::Concat_879 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Concat_217 for ONNX node: Concat_217 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Reshape_350 for ONNX tensor: onnx::Reshape_350 [05/12/2023-15:13:48] [V] [TRT] Concat_217 [Concat] outputs: [onnx::Reshape_350 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Reshape_218 [Reshape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_339 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Reshape_350 [05/12/2023-15:13:48] [V] [TRT] Reshape_218 [Reshape] inputs: [onnx::Shape_339 -> (-1, -1, 12, 32)[FLOAT]], [onnx::Reshape_350 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Reshape_218 for ONNX node: Reshape_218 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::MatMul_351 for ONNX tensor: onnx::MatMul_351 [05/12/2023-15:13:48] [V] [TRT] Reshape_218 [Reshape] outputs: [onnx::MatMul_351 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_219 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_351 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_880 [05/12/2023-15:13:48] [V] [TRT] MatMul_219 [MatMul] inputs: [onnx::MatMul_351 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_880 -> (384, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::MatMul_880 for ONNX node: onnx::MatMul_880 [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_219 for ONNX node: MatMul_219 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_353 for ONNX tensor: onnx::Add_353 [05/12/2023-15:13:48] [V] [TRT] MatMul_219 [MatMul] outputs: [onnx::Add_353 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_220 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.1.attention.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_353 [05/12/2023-15:13:48] [V] [TRT] Add_220 [Add] inputs: [encoder.layer.1.attention.output.dense.bias -> (384)[FLOAT]], [onnx::Add_353 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.1.attention.output.dense.bias for ONNX node: encoder.layer.1.attention.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_220 for ONNX node: Add_220 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: input.44 for ONNX tensor: input.44 [05/12/2023-15:13:48] [V] [TRT] Add_220 [Add] outputs: [input.44 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_221 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.44 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_278 [05/12/2023-15:13:48] [V] [TRT] Add_221 [Add] inputs: [input.44 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_278 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_221 for ONNX node: Add_221 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: input.48 for ONNX tensor: input.48 [05/12/2023-15:13:48] [V] [TRT] Add_221 [Add] outputs: [input.48 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: ReduceMean_222 [ReduceMean] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.48 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_222 [ReduceMean] inputs: [input.48 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: ReduceMean_222 for ONNX node: ReduceMean_222 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Sub_356 for ONNX tensor: onnx::Sub_356 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_222 [ReduceMean] outputs: [onnx::Sub_356 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Sub_223 [Sub] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.48 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Sub_356 [05/12/2023-15:13:48] [V] [TRT] Sub_223 [Sub] inputs: [input.48 -> (-1, -1, 384)[FLOAT]], [onnx::Sub_356 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Sub_223 for ONNX node: Sub_223 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Pow_357 for ONNX tensor: onnx::Pow_357 [05/12/2023-15:13:48] [V] [TRT] Sub_223 [Sub] outputs: [onnx::Pow_357 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_224 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_224 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_224 [Constant] outputs: [onnx::Pow_358 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Pow_225 [Pow] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Pow_357 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Pow_358 [05/12/2023-15:13:48] [V] [TRT] Pow_225 [Pow] inputs: [onnx::Pow_357 -> (-1, -1, 384)[FLOAT]], [onnx::Pow_358 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Pow_358 for ONNX node: onnx::Pow_358 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Pow_225 for ONNX node: Pow_225 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::ReduceMean_359 for ONNX tensor: onnx::ReduceMean_359 [05/12/2023-15:13:48] [V] [TRT] Pow_225 [Pow] outputs: [onnx::ReduceMean_359 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: ReduceMean_226 [ReduceMean] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::ReduceMean_359 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_226 [ReduceMean] inputs: [onnx::ReduceMean_359 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: ReduceMean_226 for ONNX node: ReduceMean_226 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_360 for ONNX tensor: onnx::Add_360 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_226 [ReduceMean] outputs: [onnx::Add_360 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_227 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_227 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_227 [Constant] outputs: [onnx::Add_361 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_228 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_360 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_361 [05/12/2023-15:13:48] [V] [TRT] Add_228 [Add] inputs: [onnx::Add_360 -> (-1, -1, 1)[FLOAT]], [onnx::Add_361 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Add_361 for ONNX node: onnx::Add_361 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_228 for ONNX node: Add_228 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Sqrt_362 for ONNX tensor: onnx::Sqrt_362 [05/12/2023-15:13:48] [V] [TRT] Add_228 [Add] outputs: [onnx::Sqrt_362 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Sqrt_229 [Sqrt] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Sqrt_362 [05/12/2023-15:13:48] [V] [TRT] Sqrt_229 [Sqrt] inputs: [onnx::Sqrt_362 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Sqrt_229 for ONNX node: Sqrt_229 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Div_363 for ONNX tensor: onnx::Div_363 [05/12/2023-15:13:48] [V] [TRT] Sqrt_229 [Sqrt] outputs: [onnx::Div_363 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Div_230 [Div] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Pow_357 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Div_363 [05/12/2023-15:13:48] [V] [TRT] Div_230 [Div] inputs: [onnx::Pow_357 -> (-1, -1, 384)[FLOAT]], [onnx::Div_363 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Div_230 for ONNX node: Div_230 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Mul_364 for ONNX tensor: onnx::Mul_364 [05/12/2023-15:13:48] [V] [TRT] Div_230 [Div] outputs: [onnx::Mul_364 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Mul_231 [Mul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Mul_364 [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.1.attention.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Mul_231 [Mul] inputs: [onnx::Mul_364 -> (-1, -1, 384)[FLOAT]], [encoder.layer.1.attention.output.LayerNorm.weight -> (384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.1.attention.output.LayerNorm.weight for ONNX node: encoder.layer.1.attention.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Registering layer: Mul_231 for ONNX node: Mul_231 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_365 for ONNX tensor: onnx::Add_365 [05/12/2023-15:13:48] [V] [TRT] Mul_231 [Mul] outputs: [onnx::Add_365 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_232 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_365 [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.1.attention.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Add_232 [Add] inputs: [onnx::Add_365 -> (-1, -1, 384)[FLOAT]], [encoder.layer.1.attention.output.LayerNorm.bias -> (384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.1.attention.output.LayerNorm.bias for ONNX node: encoder.layer.1.attention.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_232 for ONNX node: Add_232 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: attention_output.3 for ONNX tensor: attention_output.3 [05/12/2023-15:13:48] [V] [TRT] Add_232 [Add] outputs: [attention_output.3 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_233 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: attention_output.3 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_881 [05/12/2023-15:13:48] [V] [TRT] MatMul_233 [MatMul] inputs: [attention_output.3 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_881 -> (384, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::MatMul_881 for ONNX node: onnx::MatMul_881 [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_233 for ONNX node: MatMul_233 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_368 for ONNX tensor: onnx::Add_368 [05/12/2023-15:13:48] [V] [TRT] MatMul_233 [MatMul] outputs: [onnx::Add_368 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_234 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.1.intermediate.dense.bias [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_368 [05/12/2023-15:13:48] [V] [TRT] Add_234 [Add] inputs: [encoder.layer.1.intermediate.dense.bias -> (1536)[FLOAT]], [onnx::Add_368 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.1.intermediate.dense.bias for ONNX node: encoder.layer.1.intermediate.dense.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_234 for ONNX node: Add_234 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Div_369 for ONNX tensor: onnx::Div_369 [05/12/2023-15:13:48] [V] [TRT] Add_234 [Add] outputs: [onnx::Div_369 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_235 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_235 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_235 [Constant] outputs: [onnx::Div_370 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Div_236 [Div] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Div_369 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Div_370 [05/12/2023-15:13:48] [V] [TRT] Div_236 [Div] inputs: [onnx::Div_369 -> (-1, -1, 1536)[FLOAT]], [onnx::Div_370 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Div_370 for ONNX node: onnx::Div_370 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Div_236 for ONNX node: Div_236 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Erf_371 for ONNX tensor: onnx::Erf_371 [05/12/2023-15:13:48] [V] [TRT] Div_236 [Div] outputs: [onnx::Erf_371 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Erf_237 [Erf] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Erf_371 [05/12/2023-15:13:48] [V] [TRT] Erf_237 [Erf] inputs: [onnx::Erf_371 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Erf_237 for ONNX node: Erf_237 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_372 for ONNX tensor: onnx::Add_372 [05/12/2023-15:13:48] [V] [TRT] Erf_237 [Erf] outputs: [onnx::Add_372 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_238 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_238 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_238 [Constant] outputs: [onnx::Add_373 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_239 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_372 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_373 [05/12/2023-15:13:48] [V] [TRT] Add_239 [Add] inputs: [onnx::Add_372 -> (-1, -1, 1536)[FLOAT]], [onnx::Add_373 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Add_373 for ONNX node: onnx::Add_373 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_239 for ONNX node: Add_239 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Mul_374 for ONNX tensor: onnx::Mul_374 [05/12/2023-15:13:48] [V] [TRT] Add_239 [Add] outputs: [onnx::Mul_374 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Mul_240 [Mul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Div_369 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Mul_374 [05/12/2023-15:13:48] [V] [TRT] Mul_240 [Mul] inputs: [onnx::Div_369 -> (-1, -1, 1536)[FLOAT]], [onnx::Mul_374 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Mul_240 for ONNX node: Mul_240 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Mul_375 for ONNX tensor: onnx::Mul_375 [05/12/2023-15:13:48] [V] [TRT] Mul_240 [Mul] outputs: [onnx::Mul_375 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_241 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_241 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_241 [Constant] outputs: [onnx::Mul_376 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Mul_242 [Mul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Mul_375 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Mul_376 [05/12/2023-15:13:48] [V] [TRT] Mul_242 [Mul] inputs: [onnx::Mul_375 -> (-1, -1, 1536)[FLOAT]], [onnx::Mul_376 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Mul_376 for ONNX node: onnx::Mul_376 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Mul_242 for ONNX node: Mul_242 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::MatMul_377 for ONNX tensor: onnx::MatMul_377 [05/12/2023-15:13:48] [V] [TRT] Mul_242 [Mul] outputs: [onnx::MatMul_377 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_243 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_377 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_882 [05/12/2023-15:13:48] [V] [TRT] MatMul_243 [MatMul] inputs: [onnx::MatMul_377 -> (-1, -1, 1536)[FLOAT]], [onnx::MatMul_882 -> (1536, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::MatMul_882 for ONNX node: onnx::MatMul_882 [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_243 for ONNX node: MatMul_243 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_379 for ONNX tensor: onnx::Add_379 [05/12/2023-15:13:48] [V] [TRT] MatMul_243 [MatMul] outputs: [onnx::Add_379 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_244 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.1.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_379 [05/12/2023-15:13:48] [V] [TRT] Add_244 [Add] inputs: [encoder.layer.1.output.dense.bias -> (384)[FLOAT]], [onnx::Add_379 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.1.output.dense.bias for ONNX node: encoder.layer.1.output.dense.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_244 for ONNX node: Add_244 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: input.52 for ONNX tensor: input.52 [05/12/2023-15:13:48] [V] [TRT] Add_244 [Add] outputs: [input.52 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_245 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.52 [05/12/2023-15:13:48] [V] [TRT] Searching for input: attention_output.3 [05/12/2023-15:13:48] [V] [TRT] Add_245 [Add] inputs: [input.52 -> (-1, -1, 384)[FLOAT]], [attention_output.3 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_245 for ONNX node: Add_245 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: input.56 for ONNX tensor: input.56 [05/12/2023-15:13:48] [V] [TRT] Add_245 [Add] outputs: [input.56 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: ReduceMean_246 [ReduceMean] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.56 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_246 [ReduceMean] inputs: [input.56 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: ReduceMean_246 for ONNX node: ReduceMean_246 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Sub_382 for ONNX tensor: onnx::Sub_382 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_246 [ReduceMean] outputs: [onnx::Sub_382 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Sub_247 [Sub] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.56 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Sub_382 [05/12/2023-15:13:48] [V] [TRT] Sub_247 [Sub] inputs: [input.56 -> (-1, -1, 384)[FLOAT]], [onnx::Sub_382 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Sub_247 for ONNX node: Sub_247 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Pow_383 for ONNX tensor: onnx::Pow_383 [05/12/2023-15:13:48] [V] [TRT] Sub_247 [Sub] outputs: [onnx::Pow_383 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_248 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_248 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_248 [Constant] outputs: [onnx::Pow_384 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Pow_249 [Pow] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Pow_383 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Pow_384 [05/12/2023-15:13:48] [V] [TRT] Pow_249 [Pow] inputs: [onnx::Pow_383 -> (-1, -1, 384)[FLOAT]], [onnx::Pow_384 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Pow_384 for ONNX node: onnx::Pow_384 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Pow_249 for ONNX node: Pow_249 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::ReduceMean_385 for ONNX tensor: onnx::ReduceMean_385 [05/12/2023-15:13:48] [V] [TRT] Pow_249 [Pow] outputs: [onnx::ReduceMean_385 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: ReduceMean_250 [ReduceMean] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::ReduceMean_385 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_250 [ReduceMean] inputs: [onnx::ReduceMean_385 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: ReduceMean_250 for ONNX node: ReduceMean_250 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_386 for ONNX tensor: onnx::Add_386 [05/12/2023-15:13:48] [V] [TRT] ReduceMean_250 [ReduceMean] outputs: [onnx::Add_386 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_251 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_251 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_251 [Constant] outputs: [onnx::Add_387 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_252 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_386 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_387 [05/12/2023-15:13:48] [V] [TRT] Add_252 [Add] inputs: [onnx::Add_386 -> (-1, -1, 1)[FLOAT]], [onnx::Add_387 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Add_387 for ONNX node: onnx::Add_387 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_252 for ONNX node: Add_252 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Sqrt_388 for ONNX tensor: onnx::Sqrt_388 [05/12/2023-15:13:48] [V] [TRT] Add_252 [Add] outputs: [onnx::Sqrt_388 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Sqrt_253 [Sqrt] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Sqrt_388 [05/12/2023-15:13:48] [V] [TRT] Sqrt_253 [Sqrt] inputs: [onnx::Sqrt_388 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Sqrt_253 for ONNX node: Sqrt_253 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Div_389 for ONNX tensor: onnx::Div_389 [05/12/2023-15:13:48] [V] [TRT] Sqrt_253 [Sqrt] outputs: [onnx::Div_389 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Div_254 [Div] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Pow_383 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Div_389 [05/12/2023-15:13:48] [V] [TRT] Div_254 [Div] inputs: [onnx::Pow_383 -> (-1, -1, 384)[FLOAT]], [onnx::Div_389 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Div_254 for ONNX node: Div_254 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Mul_390 for ONNX tensor: onnx::Mul_390 [05/12/2023-15:13:48] [V] [TRT] Div_254 [Div] outputs: [onnx::Mul_390 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Mul_255 [Mul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Mul_390 [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.1.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Mul_255 [Mul] inputs: [onnx::Mul_390 -> (-1, -1, 384)[FLOAT]], [encoder.layer.1.output.LayerNorm.weight -> (384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.1.output.LayerNorm.weight for ONNX node: encoder.layer.1.output.LayerNorm.weight [05/12/2023-15:13:48] [V] [TRT] Registering layer: Mul_255 for ONNX node: Mul_255 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_391 for ONNX tensor: onnx::Add_391 [05/12/2023-15:13:48] [V] [TRT] Mul_255 [Mul] outputs: [onnx::Add_391 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_256 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_391 [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.1.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Add_256 [Add] inputs: [onnx::Add_391 -> (-1, -1, 384)[FLOAT]], [encoder.layer.1.output.LayerNorm.bias -> (384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.1.output.LayerNorm.bias for ONNX node: encoder.layer.1.output.LayerNorm.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_256 for ONNX node: Add_256 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::MatMul_392 for ONNX tensor: onnx::MatMul_392 [05/12/2023-15:13:48] [V] [TRT] Add_256 [Add] outputs: [onnx::MatMul_392 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_257 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_392 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_883 [05/12/2023-15:13:48] [V] [TRT] MatMul_257 [MatMul] inputs: [onnx::MatMul_392 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_883 -> (384, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::MatMul_883 for ONNX node: onnx::MatMul_883 [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_257 for ONNX node: MatMul_257 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_394 for ONNX tensor: onnx::Add_394 [05/12/2023-15:13:48] [V] [TRT] MatMul_257 [MatMul] outputs: [onnx::Add_394 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_258 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.2.attention.self.query.bias [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_394 [05/12/2023-15:13:48] [V] [TRT] Add_258 [Add] inputs: [encoder.layer.2.attention.self.query.bias -> (384)[FLOAT]], [onnx::Add_394 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.2.attention.self.query.bias for ONNX node: encoder.layer.2.attention.self.query.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_258 for ONNX node: Add_258 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: mixed_query_layer.7 for ONNX tensor: mixed_query_layer.7 [05/12/2023-15:13:48] [V] [TRT] Add_258 [Add] outputs: [mixed_query_layer.7 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_259 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_392 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_884 [05/12/2023-15:13:48] [V] [TRT] MatMul_259 [MatMul] inputs: [onnx::MatMul_392 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_884 -> (384, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::MatMul_884 for ONNX node: onnx::MatMul_884 [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_259 for ONNX node: MatMul_259 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_397 for ONNX tensor: onnx::Add_397 [05/12/2023-15:13:48] [V] [TRT] MatMul_259 [MatMul] outputs: [onnx::Add_397 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_260 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.2.attention.self.key.bias [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_397 [05/12/2023-15:13:48] [V] [TRT] Add_260 [Add] inputs: [encoder.layer.2.attention.self.key.bias -> (384)[FLOAT]], [onnx::Add_397 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.2.attention.self.key.bias for ONNX node: encoder.layer.2.attention.self.key.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_260 for ONNX node: Add_260 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Shape_398 for ONNX tensor: onnx::Shape_398 [05/12/2023-15:13:48] [V] [TRT] Add_260 [Add] outputs: [onnx::Shape_398 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_261 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_398 [05/12/2023-15:13:48] [V] [TRT] Shape_261 [Shape] inputs: [onnx::Shape_398 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_261 for ONNX node: Shape_261 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_399 for ONNX tensor: onnx::Gather_399 [05/12/2023-15:13:48] [V] [TRT] Shape_261 [Shape] outputs: [onnx::Gather_399 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_262 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_262 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_262 [Constant] outputs: [onnx::Gather_400 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_263 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_399 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_400 [05/12/2023-15:13:48] [V] [TRT] Gather_263 [Gather] inputs: [onnx::Gather_399 -> (3)[INT32]], [onnx::Gather_400 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_400 for ONNX node: onnx::Gather_400 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_263 for ONNX node: Gather_263 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_401 for ONNX tensor: onnx::Unsqueeze_401 [05/12/2023-15:13:48] [V] [TRT] Gather_263 [Gather] outputs: [onnx::Unsqueeze_401 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_264 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_398 [05/12/2023-15:13:48] [V] [TRT] Shape_264 [Shape] inputs: [onnx::Shape_398 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_264 for ONNX node: Shape_264 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_402 for ONNX tensor: onnx::Gather_402 [05/12/2023-15:13:48] [V] [TRT] Shape_264 [Shape] outputs: [onnx::Gather_402 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_265 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_265 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_265 [Constant] outputs: [onnx::Gather_403 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_266 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_402 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_403 [05/12/2023-15:13:48] [V] [TRT] Gather_266 [Gather] inputs: [onnx::Gather_402 -> (3)[INT32]], [onnx::Gather_403 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_403 for ONNX node: onnx::Gather_403 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_266 for ONNX node: Gather_266 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_404 for ONNX tensor: onnx::Unsqueeze_404 [05/12/2023-15:13:48] [V] [TRT] Gather_266 [Gather] outputs: [onnx::Unsqueeze_404 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_267 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_401 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_267 [Unsqueeze] inputs: [onnx::Unsqueeze_401 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_267 for ONNX node: Unsqueeze_267 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_407 for ONNX tensor: onnx::Concat_407 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_267 [Unsqueeze] outputs: [onnx::Concat_407 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_268 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_404 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_268 [Unsqueeze] inputs: [onnx::Unsqueeze_404 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_268 for ONNX node: Unsqueeze_268 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_408 for ONNX tensor: onnx::Concat_408 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_268 [Unsqueeze] outputs: [onnx::Concat_408 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_269 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_269 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_269 [Constant] outputs: [onnx::Concat_885 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_270 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_270 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_270 [Constant] outputs: [onnx::Concat_886 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Concat_271 [Concat] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_407 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_408 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_885 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_886 [05/12/2023-15:13:48] [V] [TRT] Concat_271 [Concat] inputs: [onnx::Concat_407 -> (1)[INT32]], [onnx::Concat_408 -> (1)[INT32]], [onnx::Concat_885 -> (1)[INT32]], [onnx::Concat_886 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_885 for ONNX node: onnx::Concat_885 [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_886 for ONNX node: onnx::Concat_886 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Concat_271 for ONNX node: Concat_271 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Reshape_411 for ONNX tensor: onnx::Reshape_411 [05/12/2023-15:13:48] [V] [TRT] Concat_271 [Concat] outputs: [onnx::Reshape_411 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Reshape_272 [Reshape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_398 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Reshape_411 [05/12/2023-15:13:48] [V] [TRT] Reshape_272 [Reshape] inputs: [onnx::Shape_398 -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_411 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Reshape_272 for ONNX node: Reshape_272 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Transpose_412 for ONNX tensor: onnx::Transpose_412 [05/12/2023-15:13:48] [V] [TRT] Reshape_272 [Reshape] outputs: [onnx::Transpose_412 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_273 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_392 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_887 [05/12/2023-15:13:48] [V] [TRT] MatMul_273 [MatMul] inputs: [onnx::MatMul_392 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_887 -> (384, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::MatMul_887 for ONNX node: onnx::MatMul_887 [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_273 for ONNX node: MatMul_273 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_414 for ONNX tensor: onnx::Add_414 [05/12/2023-15:13:48] [V] [TRT] MatMul_273 [MatMul] outputs: [onnx::Add_414 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_274 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: encoder.layer.2.attention.self.value.bias [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_414 [05/12/2023-15:13:48] [V] [TRT] Add_274 [Add] inputs: [encoder.layer.2.attention.self.value.bias -> (384)[FLOAT]], [onnx::Add_414 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: encoder.layer.2.attention.self.value.bias for ONNX node: encoder.layer.2.attention.self.value.bias [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_274 for ONNX node: Add_274 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Shape_415 for ONNX tensor: onnx::Shape_415 [05/12/2023-15:13:48] [V] [TRT] Add_274 [Add] outputs: [onnx::Shape_415 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_275 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_415 [05/12/2023-15:13:48] [V] [TRT] Shape_275 [Shape] inputs: [onnx::Shape_415 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_275 for ONNX node: Shape_275 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_416 for ONNX tensor: onnx::Gather_416 [05/12/2023-15:13:48] [V] [TRT] Shape_275 [Shape] outputs: [onnx::Gather_416 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_276 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_276 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_276 [Constant] outputs: [onnx::Gather_417 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_277 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_416 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_417 [05/12/2023-15:13:48] [V] [TRT] Gather_277 [Gather] inputs: [onnx::Gather_416 -> (3)[INT32]], [onnx::Gather_417 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_417 for ONNX node: onnx::Gather_417 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_277 for ONNX node: Gather_277 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_418 for ONNX tensor: onnx::Unsqueeze_418 [05/12/2023-15:13:48] [V] [TRT] Gather_277 [Gather] outputs: [onnx::Unsqueeze_418 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_278 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_415 [05/12/2023-15:13:48] [V] [TRT] Shape_278 [Shape] inputs: [onnx::Shape_415 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_278 for ONNX node: Shape_278 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_419 for ONNX tensor: onnx::Gather_419 [05/12/2023-15:13:48] [V] [TRT] Shape_278 [Shape] outputs: [onnx::Gather_419 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_279 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_279 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_279 [Constant] outputs: [onnx::Gather_420 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_280 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_419 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_420 [05/12/2023-15:13:48] [V] [TRT] Gather_280 [Gather] inputs: [onnx::Gather_419 -> (3)[INT32]], [onnx::Gather_420 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_420 for ONNX node: onnx::Gather_420 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_280 for ONNX node: Gather_280 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_421 for ONNX tensor: onnx::Unsqueeze_421 [05/12/2023-15:13:48] [V] [TRT] Gather_280 [Gather] outputs: [onnx::Unsqueeze_421 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_281 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_418 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_281 [Unsqueeze] inputs: [onnx::Unsqueeze_418 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_281 for ONNX node: Unsqueeze_281 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_424 for ONNX tensor: onnx::Concat_424 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_281 [Unsqueeze] outputs: [onnx::Concat_424 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_282 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_421 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_282 [Unsqueeze] inputs: [onnx::Unsqueeze_421 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_282 for ONNX node: Unsqueeze_282 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_425 for ONNX tensor: onnx::Concat_425 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_282 [Unsqueeze] outputs: [onnx::Concat_425 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_283 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_283 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_283 [Constant] outputs: [onnx::Concat_888 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_284 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_284 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_284 [Constant] outputs: [onnx::Concat_889 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Concat_285 [Concat] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_424 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_425 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_888 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_889 [05/12/2023-15:13:48] [V] [TRT] Concat_285 [Concat] inputs: [onnx::Concat_424 -> (1)[INT32]], [onnx::Concat_425 -> (1)[INT32]], [onnx::Concat_888 -> (1)[INT32]], [onnx::Concat_889 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_888 for ONNX node: onnx::Concat_888 [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_889 for ONNX node: onnx::Concat_889 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Concat_285 for ONNX node: Concat_285 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Reshape_428 for ONNX tensor: onnx::Reshape_428 [05/12/2023-15:13:48] [V] [TRT] Concat_285 [Concat] outputs: [onnx::Reshape_428 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Reshape_286 [Reshape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_415 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Reshape_428 [05/12/2023-15:13:48] [V] [TRT] Reshape_286 [Reshape] inputs: [onnx::Shape_415 -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_428 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Reshape_286 for ONNX node: Reshape_286 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Transpose_429 for ONNX tensor: onnx::Transpose_429 [05/12/2023-15:13:48] [V] [TRT] Reshape_286 [Reshape] outputs: [onnx::Transpose_429 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Transpose_287 [Transpose] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Transpose_429 [05/12/2023-15:13:48] [V] [TRT] Transpose_287 [Transpose] inputs: [onnx::Transpose_429 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Transpose_287 for ONNX node: Transpose_287 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::MatMul_430 for ONNX tensor: onnx::MatMul_430 [05/12/2023-15:13:48] [V] [TRT] Transpose_287 [Transpose] outputs: [onnx::MatMul_430 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_288 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: mixed_query_layer.7 [05/12/2023-15:13:48] [V] [TRT] Shape_288 [Shape] inputs: [mixed_query_layer.7 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_288 for ONNX node: Shape_288 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_431 for ONNX tensor: onnx::Gather_431 [05/12/2023-15:13:48] [V] [TRT] Shape_288 [Shape] outputs: [onnx::Gather_431 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_289 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_289 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_289 [Constant] outputs: [onnx::Gather_432 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_290 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_431 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_432 [05/12/2023-15:13:48] [V] [TRT] Gather_290 [Gather] inputs: [onnx::Gather_431 -> (3)[INT32]], [onnx::Gather_432 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_432 for ONNX node: onnx::Gather_432 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_290 for ONNX node: Gather_290 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_433 for ONNX tensor: onnx::Unsqueeze_433 [05/12/2023-15:13:48] [V] [TRT] Gather_290 [Gather] outputs: [onnx::Unsqueeze_433 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_291 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: mixed_query_layer.7 [05/12/2023-15:13:48] [V] [TRT] Shape_291 [Shape] inputs: [mixed_query_layer.7 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_291 for ONNX node: Shape_291 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_434 for ONNX tensor: onnx::Gather_434 [05/12/2023-15:13:48] [V] [TRT] Shape_291 [Shape] outputs: [onnx::Gather_434 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_292 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_292 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_292 [Constant] outputs: [onnx::Gather_435 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_293 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_434 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_435 [05/12/2023-15:13:48] [V] [TRT] Gather_293 [Gather] inputs: [onnx::Gather_434 -> (3)[INT32]], [onnx::Gather_435 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_435 for ONNX node: onnx::Gather_435 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_293 for ONNX node: Gather_293 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_436 for ONNX tensor: onnx::Unsqueeze_436 [05/12/2023-15:13:48] [V] [TRT] Gather_293 [Gather] outputs: [onnx::Unsqueeze_436 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_294 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_433 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_294 [Unsqueeze] inputs: [onnx::Unsqueeze_433 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_294 for ONNX node: Unsqueeze_294 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_439 for ONNX tensor: onnx::Concat_439 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_294 [Unsqueeze] outputs: [onnx::Concat_439 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_295 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_436 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_295 [Unsqueeze] inputs: [onnx::Unsqueeze_436 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_295 for ONNX node: Unsqueeze_295 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_440 for ONNX tensor: onnx::Concat_440 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_295 [Unsqueeze] outputs: [onnx::Concat_440 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_296 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_296 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_296 [Constant] outputs: [onnx::Concat_890 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_297 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_297 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_297 [Constant] outputs: [onnx::Concat_891 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Concat_298 [Concat] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_439 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_440 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_890 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_891 [05/12/2023-15:13:48] [V] [TRT] Concat_298 [Concat] inputs: [onnx::Concat_439 -> (1)[INT32]], [onnx::Concat_440 -> (1)[INT32]], [onnx::Concat_890 -> (1)[INT32]], [onnx::Concat_891 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_890 for ONNX node: onnx::Concat_890 [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_891 for ONNX node: onnx::Concat_891 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Concat_298 for ONNX node: Concat_298 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Reshape_443 for ONNX tensor: onnx::Reshape_443 [05/12/2023-15:13:48] [V] [TRT] Concat_298 [Concat] outputs: [onnx::Reshape_443 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Reshape_299 [Reshape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: mixed_query_layer.7 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Reshape_443 [05/12/2023-15:13:48] [V] [TRT] Reshape_299 [Reshape] inputs: [mixed_query_layer.7 -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_443 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Reshape_299 for ONNX node: Reshape_299 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Transpose_444 for ONNX tensor: onnx::Transpose_444 [05/12/2023-15:13:48] [V] [TRT] Reshape_299 [Reshape] outputs: [onnx::Transpose_444 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Transpose_300 [Transpose] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Transpose_444 [05/12/2023-15:13:48] [V] [TRT] Transpose_300 [Transpose] inputs: [onnx::Transpose_444 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Transpose_300 for ONNX node: Transpose_300 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::MatMul_445 for ONNX tensor: onnx::MatMul_445 [05/12/2023-15:13:48] [V] [TRT] Transpose_300 [Transpose] outputs: [onnx::MatMul_445 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Transpose_301 [Transpose] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Transpose_412 [05/12/2023-15:13:48] [V] [TRT] Transpose_301 [Transpose] inputs: [onnx::Transpose_412 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Transpose_301 for ONNX node: Transpose_301 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::MatMul_446 for ONNX tensor: onnx::MatMul_446 [05/12/2023-15:13:48] [V] [TRT] Transpose_301 [Transpose] outputs: [onnx::MatMul_446 -> (-1, 12, 32, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_302 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_445 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_446 [05/12/2023-15:13:48] [V] [TRT] MatMul_302 [MatMul] inputs: [onnx::MatMul_445 -> (-1, 12, -1, 32)[FLOAT]], [onnx::MatMul_446 -> (-1, 12, 32, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_302 for ONNX node: MatMul_302 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Div_447 for ONNX tensor: onnx::Div_447 [05/12/2023-15:13:48] [V] [TRT] MatMul_302 [MatMul] outputs: [onnx::Div_447 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_303 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_303 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_303 [Constant] outputs: [onnx::Div_448 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Div_304 [Div] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Div_447 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Div_448 [05/12/2023-15:13:48] [V] [TRT] Div_304 [Div] inputs: [onnx::Div_447 -> (-1, 12, -1, -1)[FLOAT]], [onnx::Div_448 -> ()[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Div_448 for ONNX node: onnx::Div_448 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Div_304 for ONNX node: Div_304 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Add_449 for ONNX tensor: onnx::Add_449 [05/12/2023-15:13:48] [V] [TRT] Div_304 [Div] outputs: [onnx::Add_449 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Add_305 [Add] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_449 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Add_138 [05/12/2023-15:13:48] [V] [TRT] Add_305 [Add] inputs: [onnx::Add_449 -> (-1, 12, -1, -1)[FLOAT]], [onnx::Add_138 -> (-1, 1, 1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Add_305 for ONNX node: Add_305 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: input.60 for ONNX tensor: input.60 [05/12/2023-15:13:48] [V] [TRT] Add_305 [Add] outputs: [input.60 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Softmax_306 [Softmax] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.60 [05/12/2023-15:13:48] [V] [TRT] Softmax_306 [Softmax] inputs: [input.60 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Softmax_306 for ONNX node: Softmax_306 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: input.64 for ONNX tensor: input.64 [05/12/2023-15:13:48] [V] [TRT] Softmax_306 [Softmax] outputs: [input.64 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_307 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: input.64 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_430 [05/12/2023-15:13:48] [V] [TRT] MatMul_307 [MatMul] inputs: [input.64 -> (-1, 12, -1, -1)[FLOAT]], [onnx::MatMul_430 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_307 for ONNX node: MatMul_307 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Transpose_452 for ONNX tensor: onnx::Transpose_452 [05/12/2023-15:13:48] [V] [TRT] MatMul_307 [MatMul] outputs: [onnx::Transpose_452 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Transpose_308 [Transpose] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Transpose_452 [05/12/2023-15:13:48] [V] [TRT] Transpose_308 [Transpose] inputs: [onnx::Transpose_452 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Transpose_308 for ONNX node: Transpose_308 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Shape_453 for ONNX tensor: onnx::Shape_453 [05/12/2023-15:13:48] [V] [TRT] Transpose_308 [Transpose] outputs: [onnx::Shape_453 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_309 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_453 [05/12/2023-15:13:48] [V] [TRT] Shape_309 [Shape] inputs: [onnx::Shape_453 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_309 for ONNX node: Shape_309 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_454 for ONNX tensor: onnx::Gather_454 [05/12/2023-15:13:48] [V] [TRT] Shape_309 [Shape] outputs: [onnx::Gather_454 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_310 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_310 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_310 [Constant] outputs: [onnx::Gather_455 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_311 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_454 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_455 [05/12/2023-15:13:48] [V] [TRT] Gather_311 [Gather] inputs: [onnx::Gather_454 -> (4)[INT32]], [onnx::Gather_455 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_455 for ONNX node: onnx::Gather_455 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_311 for ONNX node: Gather_311 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_456 for ONNX tensor: onnx::Unsqueeze_456 [05/12/2023-15:13:48] [V] [TRT] Gather_311 [Gather] outputs: [onnx::Unsqueeze_456 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Shape_312 [Shape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_453 [05/12/2023-15:13:48] [V] [TRT] Shape_312 [Shape] inputs: [onnx::Shape_453 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Shape_312 for ONNX node: Shape_312 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Gather_457 for ONNX tensor: onnx::Gather_457 [05/12/2023-15:13:48] [V] [TRT] Shape_312 [Shape] outputs: [onnx::Gather_457 -> (4)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_313 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_313 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_313 [Constant] outputs: [onnx::Gather_458 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Gather_314 [Gather] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_457 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Gather_458 [05/12/2023-15:13:48] [V] [TRT] Gather_314 [Gather] inputs: [onnx::Gather_457 -> (4)[INT32]], [onnx::Gather_458 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Gather_458 for ONNX node: onnx::Gather_458 [05/12/2023-15:13:48] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Gather_314 for ONNX node: Gather_314 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Unsqueeze_459 for ONNX tensor: onnx::Unsqueeze_459 [05/12/2023-15:13:48] [V] [TRT] Gather_314 [Gather] outputs: [onnx::Unsqueeze_459 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_315 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_456 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_315 [Unsqueeze] inputs: [onnx::Unsqueeze_456 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_315 for ONNX node: Unsqueeze_315 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_461 for ONNX tensor: onnx::Concat_461 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_315 [Unsqueeze] outputs: [onnx::Concat_461 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Unsqueeze_316 [Unsqueeze] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Unsqueeze_459 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_316 [Unsqueeze] inputs: [onnx::Unsqueeze_459 -> ()[INT32]], [05/12/2023-15:13:48] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:48] [V] [TRT] Registering layer: Unsqueeze_316 for ONNX node: Unsqueeze_316 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Concat_462 for ONNX tensor: onnx::Concat_462 [05/12/2023-15:13:48] [V] [TRT] Unsqueeze_316 [Unsqueeze] outputs: [onnx::Concat_462 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Constant_317 [Constant] [05/12/2023-15:13:48] [V] [TRT] Constant_317 [Constant] inputs: [05/12/2023-15:13:48] [V] [TRT] Constant_317 [Constant] outputs: [onnx::Concat_892 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Concat_318 [Concat] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_461 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_462 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Concat_892 [05/12/2023-15:13:48] [V] [TRT] Concat_318 [Concat] inputs: [onnx::Concat_461 -> (1)[INT32]], [onnx::Concat_462 -> (1)[INT32]], [onnx::Concat_892 -> (1)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::Concat_892 for ONNX node: onnx::Concat_892 [05/12/2023-15:13:48] [V] [TRT] Registering layer: Concat_318 for ONNX node: Concat_318 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::Reshape_464 for ONNX tensor: onnx::Reshape_464 [05/12/2023-15:13:48] [V] [TRT] Concat_318 [Concat] outputs: [onnx::Reshape_464 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: Reshape_319 [Reshape] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Shape_453 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::Reshape_464 [05/12/2023-15:13:48] [V] [TRT] Reshape_319 [Reshape] inputs: [onnx::Shape_453 -> (-1, -1, 12, 32)[FLOAT]], [onnx::Reshape_464 -> (3)[INT32]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: Reshape_319 for ONNX node: Reshape_319 [05/12/2023-15:13:48] [V] [TRT] Registering tensor: onnx::MatMul_465 for ONNX tensor: onnx::MatMul_465 [05/12/2023-15:13:48] [V] [TRT] Reshape_319 [Reshape] outputs: [onnx::MatMul_465 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Parsing node: MatMul_320 [MatMul] [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_465 [05/12/2023-15:13:48] [V] [TRT] Searching for input: onnx::MatMul_893 [05/12/2023-15:13:48] [V] [TRT] MatMul_320 [MatMul] inputs: [onnx::MatMul_465 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_893 -> (384, 384)[FLOAT]], [05/12/2023-15:13:48] [V] [TRT] Registering layer: onnx::MatMul_893 for ONNX node: onnx::MatMul_893 [05/12/2023-15:13:48] [V] [TRT] Registering layer: MatMul_320 for ONNX node: MatMul_320 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_467 for ONNX tensor: onnx::Add_467 [05/12/2023-15:13:49] [V] [TRT] MatMul_320 [MatMul] outputs: [onnx::Add_467 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_321 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.2.attention.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_467 [05/12/2023-15:13:49] [V] [TRT] Add_321 [Add] inputs: [encoder.layer.2.attention.output.dense.bias -> (384)[FLOAT]], [onnx::Add_467 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.2.attention.output.dense.bias for ONNX node: encoder.layer.2.attention.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_321 for ONNX node: Add_321 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.68 for ONNX tensor: input.68 [05/12/2023-15:13:49] [V] [TRT] Add_321 [Add] outputs: [input.68 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_322 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.68 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_392 [05/12/2023-15:13:49] [V] [TRT] Add_322 [Add] inputs: [input.68 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_392 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_322 for ONNX node: Add_322 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.72 for ONNX tensor: input.72 [05/12/2023-15:13:49] [V] [TRT] Add_322 [Add] outputs: [input.72 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: ReduceMean_323 [ReduceMean] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.72 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_323 [ReduceMean] inputs: [input.72 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: ReduceMean_323 for ONNX node: ReduceMean_323 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Sub_470 for ONNX tensor: onnx::Sub_470 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_323 [ReduceMean] outputs: [onnx::Sub_470 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Sub_324 [Sub] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.72 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Sub_470 [05/12/2023-15:13:49] [V] [TRT] Sub_324 [Sub] inputs: [input.72 -> (-1, -1, 384)[FLOAT]], [onnx::Sub_470 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Sub_324 for ONNX node: Sub_324 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Pow_471 for ONNX tensor: onnx::Pow_471 [05/12/2023-15:13:49] [V] [TRT] Sub_324 [Sub] outputs: [onnx::Pow_471 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_325 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_325 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_325 [Constant] outputs: [onnx::Pow_472 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Pow_326 [Pow] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_471 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_472 [05/12/2023-15:13:49] [V] [TRT] Pow_326 [Pow] inputs: [onnx::Pow_471 -> (-1, -1, 384)[FLOAT]], [onnx::Pow_472 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Pow_472 for ONNX node: onnx::Pow_472 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Pow_326 for ONNX node: Pow_326 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::ReduceMean_473 for ONNX tensor: onnx::ReduceMean_473 [05/12/2023-15:13:49] [V] [TRT] Pow_326 [Pow] outputs: [onnx::ReduceMean_473 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: ReduceMean_327 [ReduceMean] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::ReduceMean_473 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_327 [ReduceMean] inputs: [onnx::ReduceMean_473 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: ReduceMean_327 for ONNX node: ReduceMean_327 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_474 for ONNX tensor: onnx::Add_474 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_327 [ReduceMean] outputs: [onnx::Add_474 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_328 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_328 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_328 [Constant] outputs: [onnx::Add_475 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_329 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_474 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_475 [05/12/2023-15:13:49] [V] [TRT] Add_329 [Add] inputs: [onnx::Add_474 -> (-1, -1, 1)[FLOAT]], [onnx::Add_475 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Add_475 for ONNX node: onnx::Add_475 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_329 for ONNX node: Add_329 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Sqrt_476 for ONNX tensor: onnx::Sqrt_476 [05/12/2023-15:13:49] [V] [TRT] Add_329 [Add] outputs: [onnx::Sqrt_476 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Sqrt_330 [Sqrt] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Sqrt_476 [05/12/2023-15:13:49] [V] [TRT] Sqrt_330 [Sqrt] inputs: [onnx::Sqrt_476 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Sqrt_330 for ONNX node: Sqrt_330 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Div_477 for ONNX tensor: onnx::Div_477 [05/12/2023-15:13:49] [V] [TRT] Sqrt_330 [Sqrt] outputs: [onnx::Div_477 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Div_331 [Div] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_471 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_477 [05/12/2023-15:13:49] [V] [TRT] Div_331 [Div] inputs: [onnx::Pow_471 -> (-1, -1, 384)[FLOAT]], [onnx::Div_477 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Div_331 for ONNX node: Div_331 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Mul_478 for ONNX tensor: onnx::Mul_478 [05/12/2023-15:13:49] [V] [TRT] Div_331 [Div] outputs: [onnx::Mul_478 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Mul_332 [Mul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_478 [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.2.attention.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] Mul_332 [Mul] inputs: [onnx::Mul_478 -> (-1, -1, 384)[FLOAT]], [encoder.layer.2.attention.output.LayerNorm.weight -> (384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.2.attention.output.LayerNorm.weight for ONNX node: encoder.layer.2.attention.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] Registering layer: Mul_332 for ONNX node: Mul_332 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_479 for ONNX tensor: onnx::Add_479 [05/12/2023-15:13:49] [V] [TRT] Mul_332 [Mul] outputs: [onnx::Add_479 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_333 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_479 [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.2.attention.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] Add_333 [Add] inputs: [onnx::Add_479 -> (-1, -1, 384)[FLOAT]], [encoder.layer.2.attention.output.LayerNorm.bias -> (384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.2.attention.output.LayerNorm.bias for ONNX node: encoder.layer.2.attention.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_333 for ONNX node: Add_333 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: attention_output.7 for ONNX tensor: attention_output.7 [05/12/2023-15:13:49] [V] [TRT] Add_333 [Add] outputs: [attention_output.7 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_334 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: attention_output.7 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_894 [05/12/2023-15:13:49] [V] [TRT] MatMul_334 [MatMul] inputs: [attention_output.7 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_894 -> (384, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_894 for ONNX node: onnx::MatMul_894 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_334 for ONNX node: MatMul_334 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_482 for ONNX tensor: onnx::Add_482 [05/12/2023-15:13:49] [V] [TRT] MatMul_334 [MatMul] outputs: [onnx::Add_482 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_335 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.2.intermediate.dense.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_482 [05/12/2023-15:13:49] [V] [TRT] Add_335 [Add] inputs: [encoder.layer.2.intermediate.dense.bias -> (1536)[FLOAT]], [onnx::Add_482 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.2.intermediate.dense.bias for ONNX node: encoder.layer.2.intermediate.dense.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_335 for ONNX node: Add_335 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Div_483 for ONNX tensor: onnx::Div_483 [05/12/2023-15:13:49] [V] [TRT] Add_335 [Add] outputs: [onnx::Div_483 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_336 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_336 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_336 [Constant] outputs: [onnx::Div_484 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Div_337 [Div] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_483 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_484 [05/12/2023-15:13:49] [V] [TRT] Div_337 [Div] inputs: [onnx::Div_483 -> (-1, -1, 1536)[FLOAT]], [onnx::Div_484 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Div_484 for ONNX node: onnx::Div_484 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Div_337 for ONNX node: Div_337 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Erf_485 for ONNX tensor: onnx::Erf_485 [05/12/2023-15:13:49] [V] [TRT] Div_337 [Div] outputs: [onnx::Erf_485 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Erf_338 [Erf] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Erf_485 [05/12/2023-15:13:49] [V] [TRT] Erf_338 [Erf] inputs: [onnx::Erf_485 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Erf_338 for ONNX node: Erf_338 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_486 for ONNX tensor: onnx::Add_486 [05/12/2023-15:13:49] [V] [TRT] Erf_338 [Erf] outputs: [onnx::Add_486 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_339 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_339 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_339 [Constant] outputs: [onnx::Add_487 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_340 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_486 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_487 [05/12/2023-15:13:49] [V] [TRT] Add_340 [Add] inputs: [onnx::Add_486 -> (-1, -1, 1536)[FLOAT]], [onnx::Add_487 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Add_487 for ONNX node: onnx::Add_487 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_340 for ONNX node: Add_340 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Mul_488 for ONNX tensor: onnx::Mul_488 [05/12/2023-15:13:49] [V] [TRT] Add_340 [Add] outputs: [onnx::Mul_488 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Mul_341 [Mul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_483 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_488 [05/12/2023-15:13:49] [V] [TRT] Mul_341 [Mul] inputs: [onnx::Div_483 -> (-1, -1, 1536)[FLOAT]], [onnx::Mul_488 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Mul_341 for ONNX node: Mul_341 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Mul_489 for ONNX tensor: onnx::Mul_489 [05/12/2023-15:13:49] [V] [TRT] Mul_341 [Mul] outputs: [onnx::Mul_489 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_342 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_342 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_342 [Constant] outputs: [onnx::Mul_490 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Mul_343 [Mul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_489 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_490 [05/12/2023-15:13:49] [V] [TRT] Mul_343 [Mul] inputs: [onnx::Mul_489 -> (-1, -1, 1536)[FLOAT]], [onnx::Mul_490 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Mul_490 for ONNX node: onnx::Mul_490 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Mul_343 for ONNX node: Mul_343 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_491 for ONNX tensor: onnx::MatMul_491 [05/12/2023-15:13:49] [V] [TRT] Mul_343 [Mul] outputs: [onnx::MatMul_491 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_344 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_491 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_895 [05/12/2023-15:13:49] [V] [TRT] MatMul_344 [MatMul] inputs: [onnx::MatMul_491 -> (-1, -1, 1536)[FLOAT]], [onnx::MatMul_895 -> (1536, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_895 for ONNX node: onnx::MatMul_895 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_344 for ONNX node: MatMul_344 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_493 for ONNX tensor: onnx::Add_493 [05/12/2023-15:13:49] [V] [TRT] MatMul_344 [MatMul] outputs: [onnx::Add_493 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_345 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.2.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_493 [05/12/2023-15:13:49] [V] [TRT] Add_345 [Add] inputs: [encoder.layer.2.output.dense.bias -> (384)[FLOAT]], [onnx::Add_493 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.2.output.dense.bias for ONNX node: encoder.layer.2.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_345 for ONNX node: Add_345 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.76 for ONNX tensor: input.76 [05/12/2023-15:13:49] [V] [TRT] Add_345 [Add] outputs: [input.76 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_346 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.76 [05/12/2023-15:13:49] [V] [TRT] Searching for input: attention_output.7 [05/12/2023-15:13:49] [V] [TRT] Add_346 [Add] inputs: [input.76 -> (-1, -1, 384)[FLOAT]], [attention_output.7 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_346 for ONNX node: Add_346 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.80 for ONNX tensor: input.80 [05/12/2023-15:13:49] [V] [TRT] Add_346 [Add] outputs: [input.80 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: ReduceMean_347 [ReduceMean] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.80 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_347 [ReduceMean] inputs: [input.80 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: ReduceMean_347 for ONNX node: ReduceMean_347 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Sub_496 for ONNX tensor: onnx::Sub_496 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_347 [ReduceMean] outputs: [onnx::Sub_496 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Sub_348 [Sub] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.80 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Sub_496 [05/12/2023-15:13:49] [V] [TRT] Sub_348 [Sub] inputs: [input.80 -> (-1, -1, 384)[FLOAT]], [onnx::Sub_496 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Sub_348 for ONNX node: Sub_348 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Pow_497 for ONNX tensor: onnx::Pow_497 [05/12/2023-15:13:49] [V] [TRT] Sub_348 [Sub] outputs: [onnx::Pow_497 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_349 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_349 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_349 [Constant] outputs: [onnx::Pow_498 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Pow_350 [Pow] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_497 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_498 [05/12/2023-15:13:49] [V] [TRT] Pow_350 [Pow] inputs: [onnx::Pow_497 -> (-1, -1, 384)[FLOAT]], [onnx::Pow_498 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Pow_498 for ONNX node: onnx::Pow_498 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Pow_350 for ONNX node: Pow_350 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::ReduceMean_499 for ONNX tensor: onnx::ReduceMean_499 [05/12/2023-15:13:49] [V] [TRT] Pow_350 [Pow] outputs: [onnx::ReduceMean_499 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: ReduceMean_351 [ReduceMean] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::ReduceMean_499 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_351 [ReduceMean] inputs: [onnx::ReduceMean_499 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: ReduceMean_351 for ONNX node: ReduceMean_351 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_500 for ONNX tensor: onnx::Add_500 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_351 [ReduceMean] outputs: [onnx::Add_500 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_352 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_352 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_352 [Constant] outputs: [onnx::Add_501 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_353 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_500 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_501 [05/12/2023-15:13:49] [V] [TRT] Add_353 [Add] inputs: [onnx::Add_500 -> (-1, -1, 1)[FLOAT]], [onnx::Add_501 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Add_501 for ONNX node: onnx::Add_501 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_353 for ONNX node: Add_353 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Sqrt_502 for ONNX tensor: onnx::Sqrt_502 [05/12/2023-15:13:49] [V] [TRT] Add_353 [Add] outputs: [onnx::Sqrt_502 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Sqrt_354 [Sqrt] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Sqrt_502 [05/12/2023-15:13:49] [V] [TRT] Sqrt_354 [Sqrt] inputs: [onnx::Sqrt_502 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Sqrt_354 for ONNX node: Sqrt_354 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Div_503 for ONNX tensor: onnx::Div_503 [05/12/2023-15:13:49] [V] [TRT] Sqrt_354 [Sqrt] outputs: [onnx::Div_503 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Div_355 [Div] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_497 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_503 [05/12/2023-15:13:49] [V] [TRT] Div_355 [Div] inputs: [onnx::Pow_497 -> (-1, -1, 384)[FLOAT]], [onnx::Div_503 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Div_355 for ONNX node: Div_355 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Mul_504 for ONNX tensor: onnx::Mul_504 [05/12/2023-15:13:49] [V] [TRT] Div_355 [Div] outputs: [onnx::Mul_504 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Mul_356 [Mul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_504 [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.2.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] Mul_356 [Mul] inputs: [onnx::Mul_504 -> (-1, -1, 384)[FLOAT]], [encoder.layer.2.output.LayerNorm.weight -> (384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.2.output.LayerNorm.weight for ONNX node: encoder.layer.2.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] Registering layer: Mul_356 for ONNX node: Mul_356 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_505 for ONNX tensor: onnx::Add_505 [05/12/2023-15:13:49] [V] [TRT] Mul_356 [Mul] outputs: [onnx::Add_505 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_357 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_505 [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.2.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] Add_357 [Add] inputs: [onnx::Add_505 -> (-1, -1, 384)[FLOAT]], [encoder.layer.2.output.LayerNorm.bias -> (384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.2.output.LayerNorm.bias for ONNX node: encoder.layer.2.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_357 for ONNX node: Add_357 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_506 for ONNX tensor: onnx::MatMul_506 [05/12/2023-15:13:49] [V] [TRT] Add_357 [Add] outputs: [onnx::MatMul_506 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_358 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_506 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_896 [05/12/2023-15:13:49] [V] [TRT] MatMul_358 [MatMul] inputs: [onnx::MatMul_506 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_896 -> (384, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_896 for ONNX node: onnx::MatMul_896 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_358 for ONNX node: MatMul_358 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_508 for ONNX tensor: onnx::Add_508 [05/12/2023-15:13:49] [V] [TRT] MatMul_358 [MatMul] outputs: [onnx::Add_508 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_359 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.3.attention.self.query.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_508 [05/12/2023-15:13:49] [V] [TRT] Add_359 [Add] inputs: [encoder.layer.3.attention.self.query.bias -> (384)[FLOAT]], [onnx::Add_508 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.3.attention.self.query.bias for ONNX node: encoder.layer.3.attention.self.query.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_359 for ONNX node: Add_359 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: mixed_query_layer.11 for ONNX tensor: mixed_query_layer.11 [05/12/2023-15:13:49] [V] [TRT] Add_359 [Add] outputs: [mixed_query_layer.11 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_360 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_506 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_897 [05/12/2023-15:13:49] [V] [TRT] MatMul_360 [MatMul] inputs: [onnx::MatMul_506 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_897 -> (384, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_897 for ONNX node: onnx::MatMul_897 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_360 for ONNX node: MatMul_360 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_511 for ONNX tensor: onnx::Add_511 [05/12/2023-15:13:49] [V] [TRT] MatMul_360 [MatMul] outputs: [onnx::Add_511 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_361 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.3.attention.self.key.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_511 [05/12/2023-15:13:49] [V] [TRT] Add_361 [Add] inputs: [encoder.layer.3.attention.self.key.bias -> (384)[FLOAT]], [onnx::Add_511 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.3.attention.self.key.bias for ONNX node: encoder.layer.3.attention.self.key.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_361 for ONNX node: Add_361 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Shape_512 for ONNX tensor: onnx::Shape_512 [05/12/2023-15:13:49] [V] [TRT] Add_361 [Add] outputs: [onnx::Shape_512 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_362 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_512 [05/12/2023-15:13:49] [V] [TRT] Shape_362 [Shape] inputs: [onnx::Shape_512 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_362 for ONNX node: Shape_362 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_513 for ONNX tensor: onnx::Gather_513 [05/12/2023-15:13:49] [V] [TRT] Shape_362 [Shape] outputs: [onnx::Gather_513 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_363 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_363 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_363 [Constant] outputs: [onnx::Gather_514 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_364 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_513 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_514 [05/12/2023-15:13:49] [V] [TRT] Gather_364 [Gather] inputs: [onnx::Gather_513 -> (3)[INT32]], [onnx::Gather_514 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_514 for ONNX node: onnx::Gather_514 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_364 for ONNX node: Gather_364 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_515 for ONNX tensor: onnx::Unsqueeze_515 [05/12/2023-15:13:49] [V] [TRT] Gather_364 [Gather] outputs: [onnx::Unsqueeze_515 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_365 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_512 [05/12/2023-15:13:49] [V] [TRT] Shape_365 [Shape] inputs: [onnx::Shape_512 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_365 for ONNX node: Shape_365 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_516 for ONNX tensor: onnx::Gather_516 [05/12/2023-15:13:49] [V] [TRT] Shape_365 [Shape] outputs: [onnx::Gather_516 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_366 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_366 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_366 [Constant] outputs: [onnx::Gather_517 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_367 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_516 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_517 [05/12/2023-15:13:49] [V] [TRT] Gather_367 [Gather] inputs: [onnx::Gather_516 -> (3)[INT32]], [onnx::Gather_517 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_517 for ONNX node: onnx::Gather_517 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_367 for ONNX node: Gather_367 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_518 for ONNX tensor: onnx::Unsqueeze_518 [05/12/2023-15:13:49] [V] [TRT] Gather_367 [Gather] outputs: [onnx::Unsqueeze_518 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_368 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_515 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_368 [Unsqueeze] inputs: [onnx::Unsqueeze_515 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_368 for ONNX node: Unsqueeze_368 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_521 for ONNX tensor: onnx::Concat_521 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_368 [Unsqueeze] outputs: [onnx::Concat_521 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_369 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_518 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_369 [Unsqueeze] inputs: [onnx::Unsqueeze_518 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_369 for ONNX node: Unsqueeze_369 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_522 for ONNX tensor: onnx::Concat_522 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_369 [Unsqueeze] outputs: [onnx::Concat_522 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_370 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_370 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_370 [Constant] outputs: [onnx::Concat_898 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_371 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_371 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_371 [Constant] outputs: [onnx::Concat_899 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Concat_372 [Concat] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_521 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_522 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_898 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_899 [05/12/2023-15:13:49] [V] [TRT] Concat_372 [Concat] inputs: [onnx::Concat_521 -> (1)[INT32]], [onnx::Concat_522 -> (1)[INT32]], [onnx::Concat_898 -> (1)[INT32]], [onnx::Concat_899 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_898 for ONNX node: onnx::Concat_898 [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_899 for ONNX node: onnx::Concat_899 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Concat_372 for ONNX node: Concat_372 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Reshape_525 for ONNX tensor: onnx::Reshape_525 [05/12/2023-15:13:49] [V] [TRT] Concat_372 [Concat] outputs: [onnx::Reshape_525 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Reshape_373 [Reshape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_512 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Reshape_525 [05/12/2023-15:13:49] [V] [TRT] Reshape_373 [Reshape] inputs: [onnx::Shape_512 -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_525 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Reshape_373 for ONNX node: Reshape_373 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Transpose_526 for ONNX tensor: onnx::Transpose_526 [05/12/2023-15:13:49] [V] [TRT] Reshape_373 [Reshape] outputs: [onnx::Transpose_526 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_374 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_506 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_900 [05/12/2023-15:13:49] [V] [TRT] MatMul_374 [MatMul] inputs: [onnx::MatMul_506 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_900 -> (384, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_900 for ONNX node: onnx::MatMul_900 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_374 for ONNX node: MatMul_374 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_528 for ONNX tensor: onnx::Add_528 [05/12/2023-15:13:49] [V] [TRT] MatMul_374 [MatMul] outputs: [onnx::Add_528 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_375 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.3.attention.self.value.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_528 [05/12/2023-15:13:49] [V] [TRT] Add_375 [Add] inputs: [encoder.layer.3.attention.self.value.bias -> (384)[FLOAT]], [onnx::Add_528 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.3.attention.self.value.bias for ONNX node: encoder.layer.3.attention.self.value.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_375 for ONNX node: Add_375 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Shape_529 for ONNX tensor: onnx::Shape_529 [05/12/2023-15:13:49] [V] [TRT] Add_375 [Add] outputs: [onnx::Shape_529 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_376 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_529 [05/12/2023-15:13:49] [V] [TRT] Shape_376 [Shape] inputs: [onnx::Shape_529 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_376 for ONNX node: Shape_376 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_530 for ONNX tensor: onnx::Gather_530 [05/12/2023-15:13:49] [V] [TRT] Shape_376 [Shape] outputs: [onnx::Gather_530 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_377 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_377 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_377 [Constant] outputs: [onnx::Gather_531 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_378 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_530 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_531 [05/12/2023-15:13:49] [V] [TRT] Gather_378 [Gather] inputs: [onnx::Gather_530 -> (3)[INT32]], [onnx::Gather_531 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_531 for ONNX node: onnx::Gather_531 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_378 for ONNX node: Gather_378 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_532 for ONNX tensor: onnx::Unsqueeze_532 [05/12/2023-15:13:49] [V] [TRT] Gather_378 [Gather] outputs: [onnx::Unsqueeze_532 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_379 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_529 [05/12/2023-15:13:49] [V] [TRT] Shape_379 [Shape] inputs: [onnx::Shape_529 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_379 for ONNX node: Shape_379 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_533 for ONNX tensor: onnx::Gather_533 [05/12/2023-15:13:49] [V] [TRT] Shape_379 [Shape] outputs: [onnx::Gather_533 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_380 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_380 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_380 [Constant] outputs: [onnx::Gather_534 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_381 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_533 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_534 [05/12/2023-15:13:49] [V] [TRT] Gather_381 [Gather] inputs: [onnx::Gather_533 -> (3)[INT32]], [onnx::Gather_534 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_534 for ONNX node: onnx::Gather_534 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_381 for ONNX node: Gather_381 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_535 for ONNX tensor: onnx::Unsqueeze_535 [05/12/2023-15:13:49] [V] [TRT] Gather_381 [Gather] outputs: [onnx::Unsqueeze_535 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_382 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_532 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_382 [Unsqueeze] inputs: [onnx::Unsqueeze_532 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_382 for ONNX node: Unsqueeze_382 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_538 for ONNX tensor: onnx::Concat_538 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_382 [Unsqueeze] outputs: [onnx::Concat_538 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_383 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_535 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_383 [Unsqueeze] inputs: [onnx::Unsqueeze_535 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_383 for ONNX node: Unsqueeze_383 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_539 for ONNX tensor: onnx::Concat_539 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_383 [Unsqueeze] outputs: [onnx::Concat_539 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_384 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_384 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_384 [Constant] outputs: [onnx::Concat_901 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_385 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_385 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_385 [Constant] outputs: [onnx::Concat_902 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Concat_386 [Concat] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_538 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_539 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_901 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_902 [05/12/2023-15:13:49] [V] [TRT] Concat_386 [Concat] inputs: [onnx::Concat_538 -> (1)[INT32]], [onnx::Concat_539 -> (1)[INT32]], [onnx::Concat_901 -> (1)[INT32]], [onnx::Concat_902 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_901 for ONNX node: onnx::Concat_901 [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_902 for ONNX node: onnx::Concat_902 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Concat_386 for ONNX node: Concat_386 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Reshape_542 for ONNX tensor: onnx::Reshape_542 [05/12/2023-15:13:49] [V] [TRT] Concat_386 [Concat] outputs: [onnx::Reshape_542 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Reshape_387 [Reshape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_529 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Reshape_542 [05/12/2023-15:13:49] [V] [TRT] Reshape_387 [Reshape] inputs: [onnx::Shape_529 -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_542 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Reshape_387 for ONNX node: Reshape_387 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Transpose_543 for ONNX tensor: onnx::Transpose_543 [05/12/2023-15:13:49] [V] [TRT] Reshape_387 [Reshape] outputs: [onnx::Transpose_543 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Transpose_388 [Transpose] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Transpose_543 [05/12/2023-15:13:49] [V] [TRT] Transpose_388 [Transpose] inputs: [onnx::Transpose_543 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Transpose_388 for ONNX node: Transpose_388 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_544 for ONNX tensor: onnx::MatMul_544 [05/12/2023-15:13:49] [V] [TRT] Transpose_388 [Transpose] outputs: [onnx::MatMul_544 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_389 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: mixed_query_layer.11 [05/12/2023-15:13:49] [V] [TRT] Shape_389 [Shape] inputs: [mixed_query_layer.11 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_389 for ONNX node: Shape_389 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_545 for ONNX tensor: onnx::Gather_545 [05/12/2023-15:13:49] [V] [TRT] Shape_389 [Shape] outputs: [onnx::Gather_545 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_390 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_390 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_390 [Constant] outputs: [onnx::Gather_546 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_391 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_545 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_546 [05/12/2023-15:13:49] [V] [TRT] Gather_391 [Gather] inputs: [onnx::Gather_545 -> (3)[INT32]], [onnx::Gather_546 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_546 for ONNX node: onnx::Gather_546 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_391 for ONNX node: Gather_391 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_547 for ONNX tensor: onnx::Unsqueeze_547 [05/12/2023-15:13:49] [V] [TRT] Gather_391 [Gather] outputs: [onnx::Unsqueeze_547 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_392 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: mixed_query_layer.11 [05/12/2023-15:13:49] [V] [TRT] Shape_392 [Shape] inputs: [mixed_query_layer.11 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_392 for ONNX node: Shape_392 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_548 for ONNX tensor: onnx::Gather_548 [05/12/2023-15:13:49] [V] [TRT] Shape_392 [Shape] outputs: [onnx::Gather_548 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_393 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_393 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_393 [Constant] outputs: [onnx::Gather_549 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_394 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_548 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_549 [05/12/2023-15:13:49] [V] [TRT] Gather_394 [Gather] inputs: [onnx::Gather_548 -> (3)[INT32]], [onnx::Gather_549 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_549 for ONNX node: onnx::Gather_549 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_394 for ONNX node: Gather_394 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_550 for ONNX tensor: onnx::Unsqueeze_550 [05/12/2023-15:13:49] [V] [TRT] Gather_394 [Gather] outputs: [onnx::Unsqueeze_550 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_395 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_547 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_395 [Unsqueeze] inputs: [onnx::Unsqueeze_547 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_395 for ONNX node: Unsqueeze_395 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_553 for ONNX tensor: onnx::Concat_553 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_395 [Unsqueeze] outputs: [onnx::Concat_553 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_396 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_550 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_396 [Unsqueeze] inputs: [onnx::Unsqueeze_550 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_396 for ONNX node: Unsqueeze_396 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_554 for ONNX tensor: onnx::Concat_554 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_396 [Unsqueeze] outputs: [onnx::Concat_554 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_397 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_397 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_397 [Constant] outputs: [onnx::Concat_903 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_398 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_398 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_398 [Constant] outputs: [onnx::Concat_904 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Concat_399 [Concat] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_553 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_554 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_903 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_904 [05/12/2023-15:13:49] [V] [TRT] Concat_399 [Concat] inputs: [onnx::Concat_553 -> (1)[INT32]], [onnx::Concat_554 -> (1)[INT32]], [onnx::Concat_903 -> (1)[INT32]], [onnx::Concat_904 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_903 for ONNX node: onnx::Concat_903 [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_904 for ONNX node: onnx::Concat_904 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Concat_399 for ONNX node: Concat_399 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Reshape_557 for ONNX tensor: onnx::Reshape_557 [05/12/2023-15:13:49] [V] [TRT] Concat_399 [Concat] outputs: [onnx::Reshape_557 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Reshape_400 [Reshape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: mixed_query_layer.11 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Reshape_557 [05/12/2023-15:13:49] [V] [TRT] Reshape_400 [Reshape] inputs: [mixed_query_layer.11 -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_557 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Reshape_400 for ONNX node: Reshape_400 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Transpose_558 for ONNX tensor: onnx::Transpose_558 [05/12/2023-15:13:49] [V] [TRT] Reshape_400 [Reshape] outputs: [onnx::Transpose_558 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Transpose_401 [Transpose] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Transpose_558 [05/12/2023-15:13:49] [V] [TRT] Transpose_401 [Transpose] inputs: [onnx::Transpose_558 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Transpose_401 for ONNX node: Transpose_401 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_559 for ONNX tensor: onnx::MatMul_559 [05/12/2023-15:13:49] [V] [TRT] Transpose_401 [Transpose] outputs: [onnx::MatMul_559 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Transpose_402 [Transpose] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Transpose_526 [05/12/2023-15:13:49] [V] [TRT] Transpose_402 [Transpose] inputs: [onnx::Transpose_526 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Transpose_402 for ONNX node: Transpose_402 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_560 for ONNX tensor: onnx::MatMul_560 [05/12/2023-15:13:49] [V] [TRT] Transpose_402 [Transpose] outputs: [onnx::MatMul_560 -> (-1, 12, 32, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_403 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_559 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_560 [05/12/2023-15:13:49] [V] [TRT] MatMul_403 [MatMul] inputs: [onnx::MatMul_559 -> (-1, 12, -1, 32)[FLOAT]], [onnx::MatMul_560 -> (-1, 12, 32, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_403 for ONNX node: MatMul_403 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Div_561 for ONNX tensor: onnx::Div_561 [05/12/2023-15:13:49] [V] [TRT] MatMul_403 [MatMul] outputs: [onnx::Div_561 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_404 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_404 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_404 [Constant] outputs: [onnx::Div_562 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Div_405 [Div] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_561 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_562 [05/12/2023-15:13:49] [V] [TRT] Div_405 [Div] inputs: [onnx::Div_561 -> (-1, 12, -1, -1)[FLOAT]], [onnx::Div_562 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Div_562 for ONNX node: onnx::Div_562 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Div_405 for ONNX node: Div_405 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_563 for ONNX tensor: onnx::Add_563 [05/12/2023-15:13:49] [V] [TRT] Div_405 [Div] outputs: [onnx::Add_563 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_406 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_563 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_138 [05/12/2023-15:13:49] [V] [TRT] Add_406 [Add] inputs: [onnx::Add_563 -> (-1, 12, -1, -1)[FLOAT]], [onnx::Add_138 -> (-1, 1, 1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_406 for ONNX node: Add_406 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.84 for ONNX tensor: input.84 [05/12/2023-15:13:49] [V] [TRT] Add_406 [Add] outputs: [input.84 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Softmax_407 [Softmax] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.84 [05/12/2023-15:13:49] [V] [TRT] Softmax_407 [Softmax] inputs: [input.84 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Softmax_407 for ONNX node: Softmax_407 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.88 for ONNX tensor: input.88 [05/12/2023-15:13:49] [V] [TRT] Softmax_407 [Softmax] outputs: [input.88 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_408 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.88 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_544 [05/12/2023-15:13:49] [V] [TRT] MatMul_408 [MatMul] inputs: [input.88 -> (-1, 12, -1, -1)[FLOAT]], [onnx::MatMul_544 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_408 for ONNX node: MatMul_408 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Transpose_566 for ONNX tensor: onnx::Transpose_566 [05/12/2023-15:13:49] [V] [TRT] MatMul_408 [MatMul] outputs: [onnx::Transpose_566 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Transpose_409 [Transpose] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Transpose_566 [05/12/2023-15:13:49] [V] [TRT] Transpose_409 [Transpose] inputs: [onnx::Transpose_566 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Transpose_409 for ONNX node: Transpose_409 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Shape_567 for ONNX tensor: onnx::Shape_567 [05/12/2023-15:13:49] [V] [TRT] Transpose_409 [Transpose] outputs: [onnx::Shape_567 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_410 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_567 [05/12/2023-15:13:49] [V] [TRT] Shape_410 [Shape] inputs: [onnx::Shape_567 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_410 for ONNX node: Shape_410 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_568 for ONNX tensor: onnx::Gather_568 [05/12/2023-15:13:49] [V] [TRT] Shape_410 [Shape] outputs: [onnx::Gather_568 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_411 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_411 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_411 [Constant] outputs: [onnx::Gather_569 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_412 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_568 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_569 [05/12/2023-15:13:49] [V] [TRT] Gather_412 [Gather] inputs: [onnx::Gather_568 -> (4)[INT32]], [onnx::Gather_569 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_569 for ONNX node: onnx::Gather_569 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_412 for ONNX node: Gather_412 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_570 for ONNX tensor: onnx::Unsqueeze_570 [05/12/2023-15:13:49] [V] [TRT] Gather_412 [Gather] outputs: [onnx::Unsqueeze_570 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_413 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_567 [05/12/2023-15:13:49] [V] [TRT] Shape_413 [Shape] inputs: [onnx::Shape_567 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_413 for ONNX node: Shape_413 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_571 for ONNX tensor: onnx::Gather_571 [05/12/2023-15:13:49] [V] [TRT] Shape_413 [Shape] outputs: [onnx::Gather_571 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_414 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_414 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_414 [Constant] outputs: [onnx::Gather_572 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_415 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_571 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_572 [05/12/2023-15:13:49] [V] [TRT] Gather_415 [Gather] inputs: [onnx::Gather_571 -> (4)[INT32]], [onnx::Gather_572 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_572 for ONNX node: onnx::Gather_572 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_415 for ONNX node: Gather_415 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_573 for ONNX tensor: onnx::Unsqueeze_573 [05/12/2023-15:13:49] [V] [TRT] Gather_415 [Gather] outputs: [onnx::Unsqueeze_573 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_416 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_570 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_416 [Unsqueeze] inputs: [onnx::Unsqueeze_570 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_416 for ONNX node: Unsqueeze_416 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_575 for ONNX tensor: onnx::Concat_575 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_416 [Unsqueeze] outputs: [onnx::Concat_575 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_417 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_573 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_417 [Unsqueeze] inputs: [onnx::Unsqueeze_573 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_417 for ONNX node: Unsqueeze_417 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_576 for ONNX tensor: onnx::Concat_576 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_417 [Unsqueeze] outputs: [onnx::Concat_576 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_418 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_418 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_418 [Constant] outputs: [onnx::Concat_905 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Concat_419 [Concat] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_575 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_576 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_905 [05/12/2023-15:13:49] [V] [TRT] Concat_419 [Concat] inputs: [onnx::Concat_575 -> (1)[INT32]], [onnx::Concat_576 -> (1)[INT32]], [onnx::Concat_905 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_905 for ONNX node: onnx::Concat_905 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Concat_419 for ONNX node: Concat_419 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Reshape_578 for ONNX tensor: onnx::Reshape_578 [05/12/2023-15:13:49] [V] [TRT] Concat_419 [Concat] outputs: [onnx::Reshape_578 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Reshape_420 [Reshape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_567 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Reshape_578 [05/12/2023-15:13:49] [V] [TRT] Reshape_420 [Reshape] inputs: [onnx::Shape_567 -> (-1, -1, 12, 32)[FLOAT]], [onnx::Reshape_578 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Reshape_420 for ONNX node: Reshape_420 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_579 for ONNX tensor: onnx::MatMul_579 [05/12/2023-15:13:49] [V] [TRT] Reshape_420 [Reshape] outputs: [onnx::MatMul_579 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_421 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_579 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_906 [05/12/2023-15:13:49] [V] [TRT] MatMul_421 [MatMul] inputs: [onnx::MatMul_579 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_906 -> (384, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_906 for ONNX node: onnx::MatMul_906 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_421 for ONNX node: MatMul_421 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_581 for ONNX tensor: onnx::Add_581 [05/12/2023-15:13:49] [V] [TRT] MatMul_421 [MatMul] outputs: [onnx::Add_581 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_422 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.3.attention.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_581 [05/12/2023-15:13:49] [V] [TRT] Add_422 [Add] inputs: [encoder.layer.3.attention.output.dense.bias -> (384)[FLOAT]], [onnx::Add_581 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.3.attention.output.dense.bias for ONNX node: encoder.layer.3.attention.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_422 for ONNX node: Add_422 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.92 for ONNX tensor: input.92 [05/12/2023-15:13:49] [V] [TRT] Add_422 [Add] outputs: [input.92 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_423 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.92 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_506 [05/12/2023-15:13:49] [V] [TRT] Add_423 [Add] inputs: [input.92 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_506 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_423 for ONNX node: Add_423 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.96 for ONNX tensor: input.96 [05/12/2023-15:13:49] [V] [TRT] Add_423 [Add] outputs: [input.96 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: ReduceMean_424 [ReduceMean] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.96 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_424 [ReduceMean] inputs: [input.96 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: ReduceMean_424 for ONNX node: ReduceMean_424 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Sub_584 for ONNX tensor: onnx::Sub_584 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_424 [ReduceMean] outputs: [onnx::Sub_584 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Sub_425 [Sub] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.96 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Sub_584 [05/12/2023-15:13:49] [V] [TRT] Sub_425 [Sub] inputs: [input.96 -> (-1, -1, 384)[FLOAT]], [onnx::Sub_584 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Sub_425 for ONNX node: Sub_425 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Pow_585 for ONNX tensor: onnx::Pow_585 [05/12/2023-15:13:49] [V] [TRT] Sub_425 [Sub] outputs: [onnx::Pow_585 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_426 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_426 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_426 [Constant] outputs: [onnx::Pow_586 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Pow_427 [Pow] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_585 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_586 [05/12/2023-15:13:49] [V] [TRT] Pow_427 [Pow] inputs: [onnx::Pow_585 -> (-1, -1, 384)[FLOAT]], [onnx::Pow_586 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Pow_586 for ONNX node: onnx::Pow_586 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Pow_427 for ONNX node: Pow_427 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::ReduceMean_587 for ONNX tensor: onnx::ReduceMean_587 [05/12/2023-15:13:49] [V] [TRT] Pow_427 [Pow] outputs: [onnx::ReduceMean_587 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: ReduceMean_428 [ReduceMean] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::ReduceMean_587 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_428 [ReduceMean] inputs: [onnx::ReduceMean_587 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: ReduceMean_428 for ONNX node: ReduceMean_428 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_588 for ONNX tensor: onnx::Add_588 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_428 [ReduceMean] outputs: [onnx::Add_588 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_429 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_429 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_429 [Constant] outputs: [onnx::Add_589 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_430 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_588 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_589 [05/12/2023-15:13:49] [V] [TRT] Add_430 [Add] inputs: [onnx::Add_588 -> (-1, -1, 1)[FLOAT]], [onnx::Add_589 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Add_589 for ONNX node: onnx::Add_589 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_430 for ONNX node: Add_430 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Sqrt_590 for ONNX tensor: onnx::Sqrt_590 [05/12/2023-15:13:49] [V] [TRT] Add_430 [Add] outputs: [onnx::Sqrt_590 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Sqrt_431 [Sqrt] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Sqrt_590 [05/12/2023-15:13:49] [V] [TRT] Sqrt_431 [Sqrt] inputs: [onnx::Sqrt_590 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Sqrt_431 for ONNX node: Sqrt_431 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Div_591 for ONNX tensor: onnx::Div_591 [05/12/2023-15:13:49] [V] [TRT] Sqrt_431 [Sqrt] outputs: [onnx::Div_591 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Div_432 [Div] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_585 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_591 [05/12/2023-15:13:49] [V] [TRT] Div_432 [Div] inputs: [onnx::Pow_585 -> (-1, -1, 384)[FLOAT]], [onnx::Div_591 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Div_432 for ONNX node: Div_432 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Mul_592 for ONNX tensor: onnx::Mul_592 [05/12/2023-15:13:49] [V] [TRT] Div_432 [Div] outputs: [onnx::Mul_592 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Mul_433 [Mul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_592 [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.3.attention.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] Mul_433 [Mul] inputs: [onnx::Mul_592 -> (-1, -1, 384)[FLOAT]], [encoder.layer.3.attention.output.LayerNorm.weight -> (384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.3.attention.output.LayerNorm.weight for ONNX node: encoder.layer.3.attention.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] Registering layer: Mul_433 for ONNX node: Mul_433 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_593 for ONNX tensor: onnx::Add_593 [05/12/2023-15:13:49] [V] [TRT] Mul_433 [Mul] outputs: [onnx::Add_593 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_434 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_593 [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.3.attention.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] Add_434 [Add] inputs: [onnx::Add_593 -> (-1, -1, 384)[FLOAT]], [encoder.layer.3.attention.output.LayerNorm.bias -> (384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.3.attention.output.LayerNorm.bias for ONNX node: encoder.layer.3.attention.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_434 for ONNX node: Add_434 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: attention_output.11 for ONNX tensor: attention_output.11 [05/12/2023-15:13:49] [V] [TRT] Add_434 [Add] outputs: [attention_output.11 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_435 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: attention_output.11 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_907 [05/12/2023-15:13:49] [V] [TRT] MatMul_435 [MatMul] inputs: [attention_output.11 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_907 -> (384, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_907 for ONNX node: onnx::MatMul_907 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_435 for ONNX node: MatMul_435 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_596 for ONNX tensor: onnx::Add_596 [05/12/2023-15:13:49] [V] [TRT] MatMul_435 [MatMul] outputs: [onnx::Add_596 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_436 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.3.intermediate.dense.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_596 [05/12/2023-15:13:49] [V] [TRT] Add_436 [Add] inputs: [encoder.layer.3.intermediate.dense.bias -> (1536)[FLOAT]], [onnx::Add_596 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.3.intermediate.dense.bias for ONNX node: encoder.layer.3.intermediate.dense.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_436 for ONNX node: Add_436 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Div_597 for ONNX tensor: onnx::Div_597 [05/12/2023-15:13:49] [V] [TRT] Add_436 [Add] outputs: [onnx::Div_597 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_437 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_437 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_437 [Constant] outputs: [onnx::Div_598 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Div_438 [Div] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_597 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_598 [05/12/2023-15:13:49] [V] [TRT] Div_438 [Div] inputs: [onnx::Div_597 -> (-1, -1, 1536)[FLOAT]], [onnx::Div_598 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Div_598 for ONNX node: onnx::Div_598 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Div_438 for ONNX node: Div_438 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Erf_599 for ONNX tensor: onnx::Erf_599 [05/12/2023-15:13:49] [V] [TRT] Div_438 [Div] outputs: [onnx::Erf_599 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Erf_439 [Erf] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Erf_599 [05/12/2023-15:13:49] [V] [TRT] Erf_439 [Erf] inputs: [onnx::Erf_599 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Erf_439 for ONNX node: Erf_439 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_600 for ONNX tensor: onnx::Add_600 [05/12/2023-15:13:49] [V] [TRT] Erf_439 [Erf] outputs: [onnx::Add_600 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_440 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_440 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_440 [Constant] outputs: [onnx::Add_601 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_441 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_600 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_601 [05/12/2023-15:13:49] [V] [TRT] Add_441 [Add] inputs: [onnx::Add_600 -> (-1, -1, 1536)[FLOAT]], [onnx::Add_601 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Add_601 for ONNX node: onnx::Add_601 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_441 for ONNX node: Add_441 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Mul_602 for ONNX tensor: onnx::Mul_602 [05/12/2023-15:13:49] [V] [TRT] Add_441 [Add] outputs: [onnx::Mul_602 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Mul_442 [Mul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_597 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_602 [05/12/2023-15:13:49] [V] [TRT] Mul_442 [Mul] inputs: [onnx::Div_597 -> (-1, -1, 1536)[FLOAT]], [onnx::Mul_602 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Mul_442 for ONNX node: Mul_442 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Mul_603 for ONNX tensor: onnx::Mul_603 [05/12/2023-15:13:49] [V] [TRT] Mul_442 [Mul] outputs: [onnx::Mul_603 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_443 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_443 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_443 [Constant] outputs: [onnx::Mul_604 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Mul_444 [Mul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_603 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_604 [05/12/2023-15:13:49] [V] [TRT] Mul_444 [Mul] inputs: [onnx::Mul_603 -> (-1, -1, 1536)[FLOAT]], [onnx::Mul_604 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Mul_604 for ONNX node: onnx::Mul_604 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Mul_444 for ONNX node: Mul_444 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_605 for ONNX tensor: onnx::MatMul_605 [05/12/2023-15:13:49] [V] [TRT] Mul_444 [Mul] outputs: [onnx::MatMul_605 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_445 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_605 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_908 [05/12/2023-15:13:49] [V] [TRT] MatMul_445 [MatMul] inputs: [onnx::MatMul_605 -> (-1, -1, 1536)[FLOAT]], [onnx::MatMul_908 -> (1536, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_908 for ONNX node: onnx::MatMul_908 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_445 for ONNX node: MatMul_445 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_607 for ONNX tensor: onnx::Add_607 [05/12/2023-15:13:49] [V] [TRT] MatMul_445 [MatMul] outputs: [onnx::Add_607 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_446 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.3.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_607 [05/12/2023-15:13:49] [V] [TRT] Add_446 [Add] inputs: [encoder.layer.3.output.dense.bias -> (384)[FLOAT]], [onnx::Add_607 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.3.output.dense.bias for ONNX node: encoder.layer.3.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_446 for ONNX node: Add_446 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.100 for ONNX tensor: input.100 [05/12/2023-15:13:49] [V] [TRT] Add_446 [Add] outputs: [input.100 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_447 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.100 [05/12/2023-15:13:49] [V] [TRT] Searching for input: attention_output.11 [05/12/2023-15:13:49] [V] [TRT] Add_447 [Add] inputs: [input.100 -> (-1, -1, 384)[FLOAT]], [attention_output.11 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_447 for ONNX node: Add_447 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.104 for ONNX tensor: input.104 [05/12/2023-15:13:49] [V] [TRT] Add_447 [Add] outputs: [input.104 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: ReduceMean_448 [ReduceMean] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.104 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_448 [ReduceMean] inputs: [input.104 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: ReduceMean_448 for ONNX node: ReduceMean_448 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Sub_610 for ONNX tensor: onnx::Sub_610 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_448 [ReduceMean] outputs: [onnx::Sub_610 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Sub_449 [Sub] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.104 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Sub_610 [05/12/2023-15:13:49] [V] [TRT] Sub_449 [Sub] inputs: [input.104 -> (-1, -1, 384)[FLOAT]], [onnx::Sub_610 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Sub_449 for ONNX node: Sub_449 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Pow_611 for ONNX tensor: onnx::Pow_611 [05/12/2023-15:13:49] [V] [TRT] Sub_449 [Sub] outputs: [onnx::Pow_611 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_450 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_450 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_450 [Constant] outputs: [onnx::Pow_612 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Pow_451 [Pow] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_611 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_612 [05/12/2023-15:13:49] [V] [TRT] Pow_451 [Pow] inputs: [onnx::Pow_611 -> (-1, -1, 384)[FLOAT]], [onnx::Pow_612 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Pow_612 for ONNX node: onnx::Pow_612 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Pow_451 for ONNX node: Pow_451 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::ReduceMean_613 for ONNX tensor: onnx::ReduceMean_613 [05/12/2023-15:13:49] [V] [TRT] Pow_451 [Pow] outputs: [onnx::ReduceMean_613 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: ReduceMean_452 [ReduceMean] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::ReduceMean_613 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_452 [ReduceMean] inputs: [onnx::ReduceMean_613 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: ReduceMean_452 for ONNX node: ReduceMean_452 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_614 for ONNX tensor: onnx::Add_614 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_452 [ReduceMean] outputs: [onnx::Add_614 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_453 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_453 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_453 [Constant] outputs: [onnx::Add_615 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_454 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_614 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_615 [05/12/2023-15:13:49] [V] [TRT] Add_454 [Add] inputs: [onnx::Add_614 -> (-1, -1, 1)[FLOAT]], [onnx::Add_615 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Add_615 for ONNX node: onnx::Add_615 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_454 for ONNX node: Add_454 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Sqrt_616 for ONNX tensor: onnx::Sqrt_616 [05/12/2023-15:13:49] [V] [TRT] Add_454 [Add] outputs: [onnx::Sqrt_616 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Sqrt_455 [Sqrt] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Sqrt_616 [05/12/2023-15:13:49] [V] [TRT] Sqrt_455 [Sqrt] inputs: [onnx::Sqrt_616 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Sqrt_455 for ONNX node: Sqrt_455 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Div_617 for ONNX tensor: onnx::Div_617 [05/12/2023-15:13:49] [V] [TRT] Sqrt_455 [Sqrt] outputs: [onnx::Div_617 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Div_456 [Div] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_611 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_617 [05/12/2023-15:13:49] [V] [TRT] Div_456 [Div] inputs: [onnx::Pow_611 -> (-1, -1, 384)[FLOAT]], [onnx::Div_617 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Div_456 for ONNX node: Div_456 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Mul_618 for ONNX tensor: onnx::Mul_618 [05/12/2023-15:13:49] [V] [TRT] Div_456 [Div] outputs: [onnx::Mul_618 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Mul_457 [Mul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_618 [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.3.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] Mul_457 [Mul] inputs: [onnx::Mul_618 -> (-1, -1, 384)[FLOAT]], [encoder.layer.3.output.LayerNorm.weight -> (384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.3.output.LayerNorm.weight for ONNX node: encoder.layer.3.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] Registering layer: Mul_457 for ONNX node: Mul_457 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_619 for ONNX tensor: onnx::Add_619 [05/12/2023-15:13:49] [V] [TRT] Mul_457 [Mul] outputs: [onnx::Add_619 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_458 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_619 [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.3.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] Add_458 [Add] inputs: [onnx::Add_619 -> (-1, -1, 384)[FLOAT]], [encoder.layer.3.output.LayerNorm.bias -> (384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.3.output.LayerNorm.bias for ONNX node: encoder.layer.3.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_458 for ONNX node: Add_458 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_620 for ONNX tensor: onnx::MatMul_620 [05/12/2023-15:13:49] [V] [TRT] Add_458 [Add] outputs: [onnx::MatMul_620 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_459 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_620 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_909 [05/12/2023-15:13:49] [V] [TRT] MatMul_459 [MatMul] inputs: [onnx::MatMul_620 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_909 -> (384, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_909 for ONNX node: onnx::MatMul_909 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_459 for ONNX node: MatMul_459 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_622 for ONNX tensor: onnx::Add_622 [05/12/2023-15:13:49] [V] [TRT] MatMul_459 [MatMul] outputs: [onnx::Add_622 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_460 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.4.attention.self.query.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_622 [05/12/2023-15:13:49] [V] [TRT] Add_460 [Add] inputs: [encoder.layer.4.attention.self.query.bias -> (384)[FLOAT]], [onnx::Add_622 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.4.attention.self.query.bias for ONNX node: encoder.layer.4.attention.self.query.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_460 for ONNX node: Add_460 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: mixed_query_layer.15 for ONNX tensor: mixed_query_layer.15 [05/12/2023-15:13:49] [V] [TRT] Add_460 [Add] outputs: [mixed_query_layer.15 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_461 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_620 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_910 [05/12/2023-15:13:49] [V] [TRT] MatMul_461 [MatMul] inputs: [onnx::MatMul_620 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_910 -> (384, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_910 for ONNX node: onnx::MatMul_910 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_461 for ONNX node: MatMul_461 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_625 for ONNX tensor: onnx::Add_625 [05/12/2023-15:13:49] [V] [TRT] MatMul_461 [MatMul] outputs: [onnx::Add_625 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_462 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.4.attention.self.key.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_625 [05/12/2023-15:13:49] [V] [TRT] Add_462 [Add] inputs: [encoder.layer.4.attention.self.key.bias -> (384)[FLOAT]], [onnx::Add_625 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.4.attention.self.key.bias for ONNX node: encoder.layer.4.attention.self.key.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_462 for ONNX node: Add_462 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Shape_626 for ONNX tensor: onnx::Shape_626 [05/12/2023-15:13:49] [V] [TRT] Add_462 [Add] outputs: [onnx::Shape_626 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_463 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_626 [05/12/2023-15:13:49] [V] [TRT] Shape_463 [Shape] inputs: [onnx::Shape_626 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_463 for ONNX node: Shape_463 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_627 for ONNX tensor: onnx::Gather_627 [05/12/2023-15:13:49] [V] [TRT] Shape_463 [Shape] outputs: [onnx::Gather_627 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_464 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_464 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_464 [Constant] outputs: [onnx::Gather_628 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_465 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_627 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_628 [05/12/2023-15:13:49] [V] [TRT] Gather_465 [Gather] inputs: [onnx::Gather_627 -> (3)[INT32]], [onnx::Gather_628 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_628 for ONNX node: onnx::Gather_628 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_465 for ONNX node: Gather_465 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_629 for ONNX tensor: onnx::Unsqueeze_629 [05/12/2023-15:13:49] [V] [TRT] Gather_465 [Gather] outputs: [onnx::Unsqueeze_629 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_466 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_626 [05/12/2023-15:13:49] [V] [TRT] Shape_466 [Shape] inputs: [onnx::Shape_626 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_466 for ONNX node: Shape_466 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_630 for ONNX tensor: onnx::Gather_630 [05/12/2023-15:13:49] [V] [TRT] Shape_466 [Shape] outputs: [onnx::Gather_630 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_467 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_467 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_467 [Constant] outputs: [onnx::Gather_631 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_468 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_630 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_631 [05/12/2023-15:13:49] [V] [TRT] Gather_468 [Gather] inputs: [onnx::Gather_630 -> (3)[INT32]], [onnx::Gather_631 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_631 for ONNX node: onnx::Gather_631 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_468 for ONNX node: Gather_468 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_632 for ONNX tensor: onnx::Unsqueeze_632 [05/12/2023-15:13:49] [V] [TRT] Gather_468 [Gather] outputs: [onnx::Unsqueeze_632 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_469 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_629 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_469 [Unsqueeze] inputs: [onnx::Unsqueeze_629 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_469 for ONNX node: Unsqueeze_469 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_635 for ONNX tensor: onnx::Concat_635 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_469 [Unsqueeze] outputs: [onnx::Concat_635 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_470 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_632 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_470 [Unsqueeze] inputs: [onnx::Unsqueeze_632 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_470 for ONNX node: Unsqueeze_470 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_636 for ONNX tensor: onnx::Concat_636 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_470 [Unsqueeze] outputs: [onnx::Concat_636 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_471 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_471 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_471 [Constant] outputs: [onnx::Concat_911 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_472 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_472 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_472 [Constant] outputs: [onnx::Concat_912 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Concat_473 [Concat] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_635 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_636 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_911 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_912 [05/12/2023-15:13:49] [V] [TRT] Concat_473 [Concat] inputs: [onnx::Concat_635 -> (1)[INT32]], [onnx::Concat_636 -> (1)[INT32]], [onnx::Concat_911 -> (1)[INT32]], [onnx::Concat_912 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_911 for ONNX node: onnx::Concat_911 [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_912 for ONNX node: onnx::Concat_912 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Concat_473 for ONNX node: Concat_473 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Reshape_639 for ONNX tensor: onnx::Reshape_639 [05/12/2023-15:13:49] [V] [TRT] Concat_473 [Concat] outputs: [onnx::Reshape_639 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Reshape_474 [Reshape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_626 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Reshape_639 [05/12/2023-15:13:49] [V] [TRT] Reshape_474 [Reshape] inputs: [onnx::Shape_626 -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_639 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Reshape_474 for ONNX node: Reshape_474 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Transpose_640 for ONNX tensor: onnx::Transpose_640 [05/12/2023-15:13:49] [V] [TRT] Reshape_474 [Reshape] outputs: [onnx::Transpose_640 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_475 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_620 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_913 [05/12/2023-15:13:49] [V] [TRT] MatMul_475 [MatMul] inputs: [onnx::MatMul_620 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_913 -> (384, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_913 for ONNX node: onnx::MatMul_913 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_475 for ONNX node: MatMul_475 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_642 for ONNX tensor: onnx::Add_642 [05/12/2023-15:13:49] [V] [TRT] MatMul_475 [MatMul] outputs: [onnx::Add_642 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_476 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.4.attention.self.value.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_642 [05/12/2023-15:13:49] [V] [TRT] Add_476 [Add] inputs: [encoder.layer.4.attention.self.value.bias -> (384)[FLOAT]], [onnx::Add_642 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.4.attention.self.value.bias for ONNX node: encoder.layer.4.attention.self.value.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_476 for ONNX node: Add_476 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Shape_643 for ONNX tensor: onnx::Shape_643 [05/12/2023-15:13:49] [V] [TRT] Add_476 [Add] outputs: [onnx::Shape_643 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_477 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_643 [05/12/2023-15:13:49] [V] [TRT] Shape_477 [Shape] inputs: [onnx::Shape_643 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_477 for ONNX node: Shape_477 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_644 for ONNX tensor: onnx::Gather_644 [05/12/2023-15:13:49] [V] [TRT] Shape_477 [Shape] outputs: [onnx::Gather_644 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_478 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_478 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_478 [Constant] outputs: [onnx::Gather_645 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_479 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_644 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_645 [05/12/2023-15:13:49] [V] [TRT] Gather_479 [Gather] inputs: [onnx::Gather_644 -> (3)[INT32]], [onnx::Gather_645 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_645 for ONNX node: onnx::Gather_645 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_479 for ONNX node: Gather_479 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_646 for ONNX tensor: onnx::Unsqueeze_646 [05/12/2023-15:13:49] [V] [TRT] Gather_479 [Gather] outputs: [onnx::Unsqueeze_646 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_480 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_643 [05/12/2023-15:13:49] [V] [TRT] Shape_480 [Shape] inputs: [onnx::Shape_643 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_480 for ONNX node: Shape_480 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_647 for ONNX tensor: onnx::Gather_647 [05/12/2023-15:13:49] [V] [TRT] Shape_480 [Shape] outputs: [onnx::Gather_647 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_481 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_481 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_481 [Constant] outputs: [onnx::Gather_648 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_482 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_647 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_648 [05/12/2023-15:13:49] [V] [TRT] Gather_482 [Gather] inputs: [onnx::Gather_647 -> (3)[INT32]], [onnx::Gather_648 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_648 for ONNX node: onnx::Gather_648 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_482 for ONNX node: Gather_482 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_649 for ONNX tensor: onnx::Unsqueeze_649 [05/12/2023-15:13:49] [V] [TRT] Gather_482 [Gather] outputs: [onnx::Unsqueeze_649 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_483 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_646 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_483 [Unsqueeze] inputs: [onnx::Unsqueeze_646 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_483 for ONNX node: Unsqueeze_483 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_652 for ONNX tensor: onnx::Concat_652 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_483 [Unsqueeze] outputs: [onnx::Concat_652 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_484 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_649 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_484 [Unsqueeze] inputs: [onnx::Unsqueeze_649 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_484 for ONNX node: Unsqueeze_484 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_653 for ONNX tensor: onnx::Concat_653 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_484 [Unsqueeze] outputs: [onnx::Concat_653 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_485 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_485 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_485 [Constant] outputs: [onnx::Concat_914 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_486 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_486 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_486 [Constant] outputs: [onnx::Concat_915 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Concat_487 [Concat] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_652 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_653 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_914 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_915 [05/12/2023-15:13:49] [V] [TRT] Concat_487 [Concat] inputs: [onnx::Concat_652 -> (1)[INT32]], [onnx::Concat_653 -> (1)[INT32]], [onnx::Concat_914 -> (1)[INT32]], [onnx::Concat_915 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_914 for ONNX node: onnx::Concat_914 [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_915 for ONNX node: onnx::Concat_915 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Concat_487 for ONNX node: Concat_487 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Reshape_656 for ONNX tensor: onnx::Reshape_656 [05/12/2023-15:13:49] [V] [TRT] Concat_487 [Concat] outputs: [onnx::Reshape_656 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Reshape_488 [Reshape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_643 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Reshape_656 [05/12/2023-15:13:49] [V] [TRT] Reshape_488 [Reshape] inputs: [onnx::Shape_643 -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_656 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Reshape_488 for ONNX node: Reshape_488 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Transpose_657 for ONNX tensor: onnx::Transpose_657 [05/12/2023-15:13:49] [V] [TRT] Reshape_488 [Reshape] outputs: [onnx::Transpose_657 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Transpose_489 [Transpose] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Transpose_657 [05/12/2023-15:13:49] [V] [TRT] Transpose_489 [Transpose] inputs: [onnx::Transpose_657 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Transpose_489 for ONNX node: Transpose_489 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_658 for ONNX tensor: onnx::MatMul_658 [05/12/2023-15:13:49] [V] [TRT] Transpose_489 [Transpose] outputs: [onnx::MatMul_658 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_490 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: mixed_query_layer.15 [05/12/2023-15:13:49] [V] [TRT] Shape_490 [Shape] inputs: [mixed_query_layer.15 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_490 for ONNX node: Shape_490 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_659 for ONNX tensor: onnx::Gather_659 [05/12/2023-15:13:49] [V] [TRT] Shape_490 [Shape] outputs: [onnx::Gather_659 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_491 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_491 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_491 [Constant] outputs: [onnx::Gather_660 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_492 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_659 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_660 [05/12/2023-15:13:49] [V] [TRT] Gather_492 [Gather] inputs: [onnx::Gather_659 -> (3)[INT32]], [onnx::Gather_660 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_660 for ONNX node: onnx::Gather_660 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_492 for ONNX node: Gather_492 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_661 for ONNX tensor: onnx::Unsqueeze_661 [05/12/2023-15:13:49] [V] [TRT] Gather_492 [Gather] outputs: [onnx::Unsqueeze_661 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_493 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: mixed_query_layer.15 [05/12/2023-15:13:49] [V] [TRT] Shape_493 [Shape] inputs: [mixed_query_layer.15 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_493 for ONNX node: Shape_493 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_662 for ONNX tensor: onnx::Gather_662 [05/12/2023-15:13:49] [V] [TRT] Shape_493 [Shape] outputs: [onnx::Gather_662 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_494 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_494 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_494 [Constant] outputs: [onnx::Gather_663 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_495 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_662 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_663 [05/12/2023-15:13:49] [V] [TRT] Gather_495 [Gather] inputs: [onnx::Gather_662 -> (3)[INT32]], [onnx::Gather_663 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_663 for ONNX node: onnx::Gather_663 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_495 for ONNX node: Gather_495 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_664 for ONNX tensor: onnx::Unsqueeze_664 [05/12/2023-15:13:49] [V] [TRT] Gather_495 [Gather] outputs: [onnx::Unsqueeze_664 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_496 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_661 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_496 [Unsqueeze] inputs: [onnx::Unsqueeze_661 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_496 for ONNX node: Unsqueeze_496 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_667 for ONNX tensor: onnx::Concat_667 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_496 [Unsqueeze] outputs: [onnx::Concat_667 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_497 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_664 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_497 [Unsqueeze] inputs: [onnx::Unsqueeze_664 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_497 for ONNX node: Unsqueeze_497 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_668 for ONNX tensor: onnx::Concat_668 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_497 [Unsqueeze] outputs: [onnx::Concat_668 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_498 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_498 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_498 [Constant] outputs: [onnx::Concat_916 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_499 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_499 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_499 [Constant] outputs: [onnx::Concat_917 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Concat_500 [Concat] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_667 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_668 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_916 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_917 [05/12/2023-15:13:49] [V] [TRT] Concat_500 [Concat] inputs: [onnx::Concat_667 -> (1)[INT32]], [onnx::Concat_668 -> (1)[INT32]], [onnx::Concat_916 -> (1)[INT32]], [onnx::Concat_917 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_916 for ONNX node: onnx::Concat_916 [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_917 for ONNX node: onnx::Concat_917 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Concat_500 for ONNX node: Concat_500 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Reshape_671 for ONNX tensor: onnx::Reshape_671 [05/12/2023-15:13:49] [V] [TRT] Concat_500 [Concat] outputs: [onnx::Reshape_671 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Reshape_501 [Reshape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: mixed_query_layer.15 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Reshape_671 [05/12/2023-15:13:49] [V] [TRT] Reshape_501 [Reshape] inputs: [mixed_query_layer.15 -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_671 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Reshape_501 for ONNX node: Reshape_501 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Transpose_672 for ONNX tensor: onnx::Transpose_672 [05/12/2023-15:13:49] [V] [TRT] Reshape_501 [Reshape] outputs: [onnx::Transpose_672 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Transpose_502 [Transpose] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Transpose_672 [05/12/2023-15:13:49] [V] [TRT] Transpose_502 [Transpose] inputs: [onnx::Transpose_672 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Transpose_502 for ONNX node: Transpose_502 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_673 for ONNX tensor: onnx::MatMul_673 [05/12/2023-15:13:49] [V] [TRT] Transpose_502 [Transpose] outputs: [onnx::MatMul_673 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Transpose_503 [Transpose] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Transpose_640 [05/12/2023-15:13:49] [V] [TRT] Transpose_503 [Transpose] inputs: [onnx::Transpose_640 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Transpose_503 for ONNX node: Transpose_503 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_674 for ONNX tensor: onnx::MatMul_674 [05/12/2023-15:13:49] [V] [TRT] Transpose_503 [Transpose] outputs: [onnx::MatMul_674 -> (-1, 12, 32, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_504 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_673 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_674 [05/12/2023-15:13:49] [V] [TRT] MatMul_504 [MatMul] inputs: [onnx::MatMul_673 -> (-1, 12, -1, 32)[FLOAT]], [onnx::MatMul_674 -> (-1, 12, 32, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_504 for ONNX node: MatMul_504 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Div_675 for ONNX tensor: onnx::Div_675 [05/12/2023-15:13:49] [V] [TRT] MatMul_504 [MatMul] outputs: [onnx::Div_675 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_505 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_505 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_505 [Constant] outputs: [onnx::Div_676 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Div_506 [Div] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_675 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_676 [05/12/2023-15:13:49] [V] [TRT] Div_506 [Div] inputs: [onnx::Div_675 -> (-1, 12, -1, -1)[FLOAT]], [onnx::Div_676 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Div_676 for ONNX node: onnx::Div_676 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Div_506 for ONNX node: Div_506 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_677 for ONNX tensor: onnx::Add_677 [05/12/2023-15:13:49] [V] [TRT] Div_506 [Div] outputs: [onnx::Add_677 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_507 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_677 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_138 [05/12/2023-15:13:49] [V] [TRT] Add_507 [Add] inputs: [onnx::Add_677 -> (-1, 12, -1, -1)[FLOAT]], [onnx::Add_138 -> (-1, 1, 1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_507 for ONNX node: Add_507 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.108 for ONNX tensor: input.108 [05/12/2023-15:13:49] [V] [TRT] Add_507 [Add] outputs: [input.108 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Softmax_508 [Softmax] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.108 [05/12/2023-15:13:49] [V] [TRT] Softmax_508 [Softmax] inputs: [input.108 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Softmax_508 for ONNX node: Softmax_508 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.112 for ONNX tensor: input.112 [05/12/2023-15:13:49] [V] [TRT] Softmax_508 [Softmax] outputs: [input.112 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_509 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.112 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_658 [05/12/2023-15:13:49] [V] [TRT] MatMul_509 [MatMul] inputs: [input.112 -> (-1, 12, -1, -1)[FLOAT]], [onnx::MatMul_658 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_509 for ONNX node: MatMul_509 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Transpose_680 for ONNX tensor: onnx::Transpose_680 [05/12/2023-15:13:49] [V] [TRT] MatMul_509 [MatMul] outputs: [onnx::Transpose_680 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Transpose_510 [Transpose] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Transpose_680 [05/12/2023-15:13:49] [V] [TRT] Transpose_510 [Transpose] inputs: [onnx::Transpose_680 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Transpose_510 for ONNX node: Transpose_510 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Shape_681 for ONNX tensor: onnx::Shape_681 [05/12/2023-15:13:49] [V] [TRT] Transpose_510 [Transpose] outputs: [onnx::Shape_681 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_511 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_681 [05/12/2023-15:13:49] [V] [TRT] Shape_511 [Shape] inputs: [onnx::Shape_681 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_511 for ONNX node: Shape_511 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_682 for ONNX tensor: onnx::Gather_682 [05/12/2023-15:13:49] [V] [TRT] Shape_511 [Shape] outputs: [onnx::Gather_682 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_512 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_512 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_512 [Constant] outputs: [onnx::Gather_683 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_513 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_682 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_683 [05/12/2023-15:13:49] [V] [TRT] Gather_513 [Gather] inputs: [onnx::Gather_682 -> (4)[INT32]], [onnx::Gather_683 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_683 for ONNX node: onnx::Gather_683 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_513 for ONNX node: Gather_513 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_684 for ONNX tensor: onnx::Unsqueeze_684 [05/12/2023-15:13:49] [V] [TRT] Gather_513 [Gather] outputs: [onnx::Unsqueeze_684 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_514 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_681 [05/12/2023-15:13:49] [V] [TRT] Shape_514 [Shape] inputs: [onnx::Shape_681 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_514 for ONNX node: Shape_514 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_685 for ONNX tensor: onnx::Gather_685 [05/12/2023-15:13:49] [V] [TRT] Shape_514 [Shape] outputs: [onnx::Gather_685 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_515 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_515 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_515 [Constant] outputs: [onnx::Gather_686 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_516 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_685 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_686 [05/12/2023-15:13:49] [V] [TRT] Gather_516 [Gather] inputs: [onnx::Gather_685 -> (4)[INT32]], [onnx::Gather_686 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_686 for ONNX node: onnx::Gather_686 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_516 for ONNX node: Gather_516 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_687 for ONNX tensor: onnx::Unsqueeze_687 [05/12/2023-15:13:49] [V] [TRT] Gather_516 [Gather] outputs: [onnx::Unsqueeze_687 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_517 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_684 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_517 [Unsqueeze] inputs: [onnx::Unsqueeze_684 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_517 for ONNX node: Unsqueeze_517 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_689 for ONNX tensor: onnx::Concat_689 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_517 [Unsqueeze] outputs: [onnx::Concat_689 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_518 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_687 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_518 [Unsqueeze] inputs: [onnx::Unsqueeze_687 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_518 for ONNX node: Unsqueeze_518 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_690 for ONNX tensor: onnx::Concat_690 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_518 [Unsqueeze] outputs: [onnx::Concat_690 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_519 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_519 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_519 [Constant] outputs: [onnx::Concat_918 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Concat_520 [Concat] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_689 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_690 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_918 [05/12/2023-15:13:49] [V] [TRT] Concat_520 [Concat] inputs: [onnx::Concat_689 -> (1)[INT32]], [onnx::Concat_690 -> (1)[INT32]], [onnx::Concat_918 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_918 for ONNX node: onnx::Concat_918 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Concat_520 for ONNX node: Concat_520 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Reshape_692 for ONNX tensor: onnx::Reshape_692 [05/12/2023-15:13:49] [V] [TRT] Concat_520 [Concat] outputs: [onnx::Reshape_692 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Reshape_521 [Reshape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_681 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Reshape_692 [05/12/2023-15:13:49] [V] [TRT] Reshape_521 [Reshape] inputs: [onnx::Shape_681 -> (-1, -1, 12, 32)[FLOAT]], [onnx::Reshape_692 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Reshape_521 for ONNX node: Reshape_521 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_693 for ONNX tensor: onnx::MatMul_693 [05/12/2023-15:13:49] [V] [TRT] Reshape_521 [Reshape] outputs: [onnx::MatMul_693 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_522 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_693 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_919 [05/12/2023-15:13:49] [V] [TRT] MatMul_522 [MatMul] inputs: [onnx::MatMul_693 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_919 -> (384, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_919 for ONNX node: onnx::MatMul_919 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_522 for ONNX node: MatMul_522 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_695 for ONNX tensor: onnx::Add_695 [05/12/2023-15:13:49] [V] [TRT] MatMul_522 [MatMul] outputs: [onnx::Add_695 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_523 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.4.attention.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_695 [05/12/2023-15:13:49] [V] [TRT] Add_523 [Add] inputs: [encoder.layer.4.attention.output.dense.bias -> (384)[FLOAT]], [onnx::Add_695 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.4.attention.output.dense.bias for ONNX node: encoder.layer.4.attention.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_523 for ONNX node: Add_523 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.116 for ONNX tensor: input.116 [05/12/2023-15:13:49] [V] [TRT] Add_523 [Add] outputs: [input.116 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_524 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.116 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_620 [05/12/2023-15:13:49] [V] [TRT] Add_524 [Add] inputs: [input.116 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_620 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_524 for ONNX node: Add_524 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.120 for ONNX tensor: input.120 [05/12/2023-15:13:49] [V] [TRT] Add_524 [Add] outputs: [input.120 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: ReduceMean_525 [ReduceMean] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.120 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_525 [ReduceMean] inputs: [input.120 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: ReduceMean_525 for ONNX node: ReduceMean_525 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Sub_698 for ONNX tensor: onnx::Sub_698 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_525 [ReduceMean] outputs: [onnx::Sub_698 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Sub_526 [Sub] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.120 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Sub_698 [05/12/2023-15:13:49] [V] [TRT] Sub_526 [Sub] inputs: [input.120 -> (-1, -1, 384)[FLOAT]], [onnx::Sub_698 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Sub_526 for ONNX node: Sub_526 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Pow_699 for ONNX tensor: onnx::Pow_699 [05/12/2023-15:13:49] [V] [TRT] Sub_526 [Sub] outputs: [onnx::Pow_699 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_527 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_527 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_527 [Constant] outputs: [onnx::Pow_700 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Pow_528 [Pow] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_699 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_700 [05/12/2023-15:13:49] [V] [TRT] Pow_528 [Pow] inputs: [onnx::Pow_699 -> (-1, -1, 384)[FLOAT]], [onnx::Pow_700 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Pow_700 for ONNX node: onnx::Pow_700 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Pow_528 for ONNX node: Pow_528 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::ReduceMean_701 for ONNX tensor: onnx::ReduceMean_701 [05/12/2023-15:13:49] [V] [TRT] Pow_528 [Pow] outputs: [onnx::ReduceMean_701 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: ReduceMean_529 [ReduceMean] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::ReduceMean_701 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_529 [ReduceMean] inputs: [onnx::ReduceMean_701 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: ReduceMean_529 for ONNX node: ReduceMean_529 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_702 for ONNX tensor: onnx::Add_702 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_529 [ReduceMean] outputs: [onnx::Add_702 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_530 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_530 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_530 [Constant] outputs: [onnx::Add_703 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_531 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_702 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_703 [05/12/2023-15:13:49] [V] [TRT] Add_531 [Add] inputs: [onnx::Add_702 -> (-1, -1, 1)[FLOAT]], [onnx::Add_703 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Add_703 for ONNX node: onnx::Add_703 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_531 for ONNX node: Add_531 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Sqrt_704 for ONNX tensor: onnx::Sqrt_704 [05/12/2023-15:13:49] [V] [TRT] Add_531 [Add] outputs: [onnx::Sqrt_704 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Sqrt_532 [Sqrt] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Sqrt_704 [05/12/2023-15:13:49] [V] [TRT] Sqrt_532 [Sqrt] inputs: [onnx::Sqrt_704 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Sqrt_532 for ONNX node: Sqrt_532 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Div_705 for ONNX tensor: onnx::Div_705 [05/12/2023-15:13:49] [V] [TRT] Sqrt_532 [Sqrt] outputs: [onnx::Div_705 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Div_533 [Div] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_699 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_705 [05/12/2023-15:13:49] [V] [TRT] Div_533 [Div] inputs: [onnx::Pow_699 -> (-1, -1, 384)[FLOAT]], [onnx::Div_705 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Div_533 for ONNX node: Div_533 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Mul_706 for ONNX tensor: onnx::Mul_706 [05/12/2023-15:13:49] [V] [TRT] Div_533 [Div] outputs: [onnx::Mul_706 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Mul_534 [Mul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_706 [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.4.attention.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] Mul_534 [Mul] inputs: [onnx::Mul_706 -> (-1, -1, 384)[FLOAT]], [encoder.layer.4.attention.output.LayerNorm.weight -> (384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.4.attention.output.LayerNorm.weight for ONNX node: encoder.layer.4.attention.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] Registering layer: Mul_534 for ONNX node: Mul_534 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_707 for ONNX tensor: onnx::Add_707 [05/12/2023-15:13:49] [V] [TRT] Mul_534 [Mul] outputs: [onnx::Add_707 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_535 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_707 [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.4.attention.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] Add_535 [Add] inputs: [onnx::Add_707 -> (-1, -1, 384)[FLOAT]], [encoder.layer.4.attention.output.LayerNorm.bias -> (384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.4.attention.output.LayerNorm.bias for ONNX node: encoder.layer.4.attention.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_535 for ONNX node: Add_535 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: attention_output.15 for ONNX tensor: attention_output.15 [05/12/2023-15:13:49] [V] [TRT] Add_535 [Add] outputs: [attention_output.15 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_536 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: attention_output.15 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_920 [05/12/2023-15:13:49] [V] [TRT] MatMul_536 [MatMul] inputs: [attention_output.15 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_920 -> (384, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_920 for ONNX node: onnx::MatMul_920 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_536 for ONNX node: MatMul_536 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_710 for ONNX tensor: onnx::Add_710 [05/12/2023-15:13:49] [V] [TRT] MatMul_536 [MatMul] outputs: [onnx::Add_710 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_537 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.4.intermediate.dense.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_710 [05/12/2023-15:13:49] [V] [TRT] Add_537 [Add] inputs: [encoder.layer.4.intermediate.dense.bias -> (1536)[FLOAT]], [onnx::Add_710 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.4.intermediate.dense.bias for ONNX node: encoder.layer.4.intermediate.dense.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_537 for ONNX node: Add_537 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Div_711 for ONNX tensor: onnx::Div_711 [05/12/2023-15:13:49] [V] [TRT] Add_537 [Add] outputs: [onnx::Div_711 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_538 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_538 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_538 [Constant] outputs: [onnx::Div_712 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Div_539 [Div] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_711 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_712 [05/12/2023-15:13:49] [V] [TRT] Div_539 [Div] inputs: [onnx::Div_711 -> (-1, -1, 1536)[FLOAT]], [onnx::Div_712 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Div_712 for ONNX node: onnx::Div_712 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Div_539 for ONNX node: Div_539 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Erf_713 for ONNX tensor: onnx::Erf_713 [05/12/2023-15:13:49] [V] [TRT] Div_539 [Div] outputs: [onnx::Erf_713 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Erf_540 [Erf] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Erf_713 [05/12/2023-15:13:49] [V] [TRT] Erf_540 [Erf] inputs: [onnx::Erf_713 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Erf_540 for ONNX node: Erf_540 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_714 for ONNX tensor: onnx::Add_714 [05/12/2023-15:13:49] [V] [TRT] Erf_540 [Erf] outputs: [onnx::Add_714 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_541 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_541 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_541 [Constant] outputs: [onnx::Add_715 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_542 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_714 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_715 [05/12/2023-15:13:49] [V] [TRT] Add_542 [Add] inputs: [onnx::Add_714 -> (-1, -1, 1536)[FLOAT]], [onnx::Add_715 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Add_715 for ONNX node: onnx::Add_715 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_542 for ONNX node: Add_542 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Mul_716 for ONNX tensor: onnx::Mul_716 [05/12/2023-15:13:49] [V] [TRT] Add_542 [Add] outputs: [onnx::Mul_716 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Mul_543 [Mul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_711 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_716 [05/12/2023-15:13:49] [V] [TRT] Mul_543 [Mul] inputs: [onnx::Div_711 -> (-1, -1, 1536)[FLOAT]], [onnx::Mul_716 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Mul_543 for ONNX node: Mul_543 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Mul_717 for ONNX tensor: onnx::Mul_717 [05/12/2023-15:13:49] [V] [TRT] Mul_543 [Mul] outputs: [onnx::Mul_717 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_544 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_544 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_544 [Constant] outputs: [onnx::Mul_718 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Mul_545 [Mul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_717 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_718 [05/12/2023-15:13:49] [V] [TRT] Mul_545 [Mul] inputs: [onnx::Mul_717 -> (-1, -1, 1536)[FLOAT]], [onnx::Mul_718 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Mul_718 for ONNX node: onnx::Mul_718 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Mul_545 for ONNX node: Mul_545 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_719 for ONNX tensor: onnx::MatMul_719 [05/12/2023-15:13:49] [V] [TRT] Mul_545 [Mul] outputs: [onnx::MatMul_719 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_546 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_719 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_921 [05/12/2023-15:13:49] [V] [TRT] MatMul_546 [MatMul] inputs: [onnx::MatMul_719 -> (-1, -1, 1536)[FLOAT]], [onnx::MatMul_921 -> (1536, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_921 for ONNX node: onnx::MatMul_921 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_546 for ONNX node: MatMul_546 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_721 for ONNX tensor: onnx::Add_721 [05/12/2023-15:13:49] [V] [TRT] MatMul_546 [MatMul] outputs: [onnx::Add_721 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_547 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.4.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_721 [05/12/2023-15:13:49] [V] [TRT] Add_547 [Add] inputs: [encoder.layer.4.output.dense.bias -> (384)[FLOAT]], [onnx::Add_721 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.4.output.dense.bias for ONNX node: encoder.layer.4.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_547 for ONNX node: Add_547 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.124 for ONNX tensor: input.124 [05/12/2023-15:13:49] [V] [TRT] Add_547 [Add] outputs: [input.124 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_548 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.124 [05/12/2023-15:13:49] [V] [TRT] Searching for input: attention_output.15 [05/12/2023-15:13:49] [V] [TRT] Add_548 [Add] inputs: [input.124 -> (-1, -1, 384)[FLOAT]], [attention_output.15 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_548 for ONNX node: Add_548 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.128 for ONNX tensor: input.128 [05/12/2023-15:13:49] [V] [TRT] Add_548 [Add] outputs: [input.128 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: ReduceMean_549 [ReduceMean] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.128 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_549 [ReduceMean] inputs: [input.128 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: ReduceMean_549 for ONNX node: ReduceMean_549 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Sub_724 for ONNX tensor: onnx::Sub_724 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_549 [ReduceMean] outputs: [onnx::Sub_724 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Sub_550 [Sub] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.128 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Sub_724 [05/12/2023-15:13:49] [V] [TRT] Sub_550 [Sub] inputs: [input.128 -> (-1, -1, 384)[FLOAT]], [onnx::Sub_724 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Sub_550 for ONNX node: Sub_550 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Pow_725 for ONNX tensor: onnx::Pow_725 [05/12/2023-15:13:49] [V] [TRT] Sub_550 [Sub] outputs: [onnx::Pow_725 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_551 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_551 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_551 [Constant] outputs: [onnx::Pow_726 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Pow_552 [Pow] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_725 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_726 [05/12/2023-15:13:49] [V] [TRT] Pow_552 [Pow] inputs: [onnx::Pow_725 -> (-1, -1, 384)[FLOAT]], [onnx::Pow_726 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Pow_726 for ONNX node: onnx::Pow_726 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Pow_552 for ONNX node: Pow_552 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::ReduceMean_727 for ONNX tensor: onnx::ReduceMean_727 [05/12/2023-15:13:49] [V] [TRT] Pow_552 [Pow] outputs: [onnx::ReduceMean_727 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: ReduceMean_553 [ReduceMean] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::ReduceMean_727 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_553 [ReduceMean] inputs: [onnx::ReduceMean_727 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: ReduceMean_553 for ONNX node: ReduceMean_553 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_728 for ONNX tensor: onnx::Add_728 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_553 [ReduceMean] outputs: [onnx::Add_728 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_554 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_554 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_554 [Constant] outputs: [onnx::Add_729 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_555 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_728 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_729 [05/12/2023-15:13:49] [V] [TRT] Add_555 [Add] inputs: [onnx::Add_728 -> (-1, -1, 1)[FLOAT]], [onnx::Add_729 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Add_729 for ONNX node: onnx::Add_729 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_555 for ONNX node: Add_555 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Sqrt_730 for ONNX tensor: onnx::Sqrt_730 [05/12/2023-15:13:49] [V] [TRT] Add_555 [Add] outputs: [onnx::Sqrt_730 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Sqrt_556 [Sqrt] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Sqrt_730 [05/12/2023-15:13:49] [V] [TRT] Sqrt_556 [Sqrt] inputs: [onnx::Sqrt_730 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Sqrt_556 for ONNX node: Sqrt_556 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Div_731 for ONNX tensor: onnx::Div_731 [05/12/2023-15:13:49] [V] [TRT] Sqrt_556 [Sqrt] outputs: [onnx::Div_731 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Div_557 [Div] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_725 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_731 [05/12/2023-15:13:49] [V] [TRT] Div_557 [Div] inputs: [onnx::Pow_725 -> (-1, -1, 384)[FLOAT]], [onnx::Div_731 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Div_557 for ONNX node: Div_557 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Mul_732 for ONNX tensor: onnx::Mul_732 [05/12/2023-15:13:49] [V] [TRT] Div_557 [Div] outputs: [onnx::Mul_732 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Mul_558 [Mul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_732 [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.4.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] Mul_558 [Mul] inputs: [onnx::Mul_732 -> (-1, -1, 384)[FLOAT]], [encoder.layer.4.output.LayerNorm.weight -> (384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.4.output.LayerNorm.weight for ONNX node: encoder.layer.4.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] Registering layer: Mul_558 for ONNX node: Mul_558 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_733 for ONNX tensor: onnx::Add_733 [05/12/2023-15:13:49] [V] [TRT] Mul_558 [Mul] outputs: [onnx::Add_733 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_559 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_733 [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.4.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] Add_559 [Add] inputs: [onnx::Add_733 -> (-1, -1, 384)[FLOAT]], [encoder.layer.4.output.LayerNorm.bias -> (384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.4.output.LayerNorm.bias for ONNX node: encoder.layer.4.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_559 for ONNX node: Add_559 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_734 for ONNX tensor: onnx::MatMul_734 [05/12/2023-15:13:49] [V] [TRT] Add_559 [Add] outputs: [onnx::MatMul_734 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_560 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_734 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_922 [05/12/2023-15:13:49] [V] [TRT] MatMul_560 [MatMul] inputs: [onnx::MatMul_734 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_922 -> (384, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_922 for ONNX node: onnx::MatMul_922 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_560 for ONNX node: MatMul_560 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_736 for ONNX tensor: onnx::Add_736 [05/12/2023-15:13:49] [V] [TRT] MatMul_560 [MatMul] outputs: [onnx::Add_736 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_561 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.5.attention.self.query.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_736 [05/12/2023-15:13:49] [V] [TRT] Add_561 [Add] inputs: [encoder.layer.5.attention.self.query.bias -> (384)[FLOAT]], [onnx::Add_736 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.5.attention.self.query.bias for ONNX node: encoder.layer.5.attention.self.query.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_561 for ONNX node: Add_561 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: mixed_query_layer.19 for ONNX tensor: mixed_query_layer.19 [05/12/2023-15:13:49] [V] [TRT] Add_561 [Add] outputs: [mixed_query_layer.19 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_562 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_734 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_923 [05/12/2023-15:13:49] [V] [TRT] MatMul_562 [MatMul] inputs: [onnx::MatMul_734 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_923 -> (384, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_923 for ONNX node: onnx::MatMul_923 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_562 for ONNX node: MatMul_562 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_739 for ONNX tensor: onnx::Add_739 [05/12/2023-15:13:49] [V] [TRT] MatMul_562 [MatMul] outputs: [onnx::Add_739 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_563 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.5.attention.self.key.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_739 [05/12/2023-15:13:49] [V] [TRT] Add_563 [Add] inputs: [encoder.layer.5.attention.self.key.bias -> (384)[FLOAT]], [onnx::Add_739 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.5.attention.self.key.bias for ONNX node: encoder.layer.5.attention.self.key.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_563 for ONNX node: Add_563 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Shape_740 for ONNX tensor: onnx::Shape_740 [05/12/2023-15:13:49] [V] [TRT] Add_563 [Add] outputs: [onnx::Shape_740 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_564 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_740 [05/12/2023-15:13:49] [V] [TRT] Shape_564 [Shape] inputs: [onnx::Shape_740 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_564 for ONNX node: Shape_564 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_741 for ONNX tensor: onnx::Gather_741 [05/12/2023-15:13:49] [V] [TRT] Shape_564 [Shape] outputs: [onnx::Gather_741 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_565 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_565 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_565 [Constant] outputs: [onnx::Gather_742 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_566 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_741 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_742 [05/12/2023-15:13:49] [V] [TRT] Gather_566 [Gather] inputs: [onnx::Gather_741 -> (3)[INT32]], [onnx::Gather_742 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_742 for ONNX node: onnx::Gather_742 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_566 for ONNX node: Gather_566 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_743 for ONNX tensor: onnx::Unsqueeze_743 [05/12/2023-15:13:49] [V] [TRT] Gather_566 [Gather] outputs: [onnx::Unsqueeze_743 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_567 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_740 [05/12/2023-15:13:49] [V] [TRT] Shape_567 [Shape] inputs: [onnx::Shape_740 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_567 for ONNX node: Shape_567 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_744 for ONNX tensor: onnx::Gather_744 [05/12/2023-15:13:49] [V] [TRT] Shape_567 [Shape] outputs: [onnx::Gather_744 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_568 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_568 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_568 [Constant] outputs: [onnx::Gather_745 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_569 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_744 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_745 [05/12/2023-15:13:49] [V] [TRT] Gather_569 [Gather] inputs: [onnx::Gather_744 -> (3)[INT32]], [onnx::Gather_745 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_745 for ONNX node: onnx::Gather_745 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_569 for ONNX node: Gather_569 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_746 for ONNX tensor: onnx::Unsqueeze_746 [05/12/2023-15:13:49] [V] [TRT] Gather_569 [Gather] outputs: [onnx::Unsqueeze_746 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_570 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_743 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_570 [Unsqueeze] inputs: [onnx::Unsqueeze_743 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_570 for ONNX node: Unsqueeze_570 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_749 for ONNX tensor: onnx::Concat_749 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_570 [Unsqueeze] outputs: [onnx::Concat_749 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_571 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_746 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_571 [Unsqueeze] inputs: [onnx::Unsqueeze_746 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_571 for ONNX node: Unsqueeze_571 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_750 for ONNX tensor: onnx::Concat_750 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_571 [Unsqueeze] outputs: [onnx::Concat_750 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_572 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_572 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_572 [Constant] outputs: [onnx::Concat_924 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_573 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_573 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_573 [Constant] outputs: [onnx::Concat_925 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Concat_574 [Concat] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_749 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_750 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_924 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_925 [05/12/2023-15:13:49] [V] [TRT] Concat_574 [Concat] inputs: [onnx::Concat_749 -> (1)[INT32]], [onnx::Concat_750 -> (1)[INT32]], [onnx::Concat_924 -> (1)[INT32]], [onnx::Concat_925 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_924 for ONNX node: onnx::Concat_924 [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_925 for ONNX node: onnx::Concat_925 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Concat_574 for ONNX node: Concat_574 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Reshape_753 for ONNX tensor: onnx::Reshape_753 [05/12/2023-15:13:49] [V] [TRT] Concat_574 [Concat] outputs: [onnx::Reshape_753 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Reshape_575 [Reshape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_740 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Reshape_753 [05/12/2023-15:13:49] [V] [TRT] Reshape_575 [Reshape] inputs: [onnx::Shape_740 -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_753 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Reshape_575 for ONNX node: Reshape_575 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Transpose_754 for ONNX tensor: onnx::Transpose_754 [05/12/2023-15:13:49] [V] [TRT] Reshape_575 [Reshape] outputs: [onnx::Transpose_754 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_576 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_734 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_926 [05/12/2023-15:13:49] [V] [TRT] MatMul_576 [MatMul] inputs: [onnx::MatMul_734 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_926 -> (384, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_926 for ONNX node: onnx::MatMul_926 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_576 for ONNX node: MatMul_576 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_756 for ONNX tensor: onnx::Add_756 [05/12/2023-15:13:49] [V] [TRT] MatMul_576 [MatMul] outputs: [onnx::Add_756 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_577 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.5.attention.self.value.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_756 [05/12/2023-15:13:49] [V] [TRT] Add_577 [Add] inputs: [encoder.layer.5.attention.self.value.bias -> (384)[FLOAT]], [onnx::Add_756 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.5.attention.self.value.bias for ONNX node: encoder.layer.5.attention.self.value.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_577 for ONNX node: Add_577 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Shape_757 for ONNX tensor: onnx::Shape_757 [05/12/2023-15:13:49] [V] [TRT] Add_577 [Add] outputs: [onnx::Shape_757 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_578 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_757 [05/12/2023-15:13:49] [V] [TRT] Shape_578 [Shape] inputs: [onnx::Shape_757 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_578 for ONNX node: Shape_578 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_758 for ONNX tensor: onnx::Gather_758 [05/12/2023-15:13:49] [V] [TRT] Shape_578 [Shape] outputs: [onnx::Gather_758 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_579 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_579 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_579 [Constant] outputs: [onnx::Gather_759 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_580 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_758 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_759 [05/12/2023-15:13:49] [V] [TRT] Gather_580 [Gather] inputs: [onnx::Gather_758 -> (3)[INT32]], [onnx::Gather_759 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_759 for ONNX node: onnx::Gather_759 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_580 for ONNX node: Gather_580 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_760 for ONNX tensor: onnx::Unsqueeze_760 [05/12/2023-15:13:49] [V] [TRT] Gather_580 [Gather] outputs: [onnx::Unsqueeze_760 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_581 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_757 [05/12/2023-15:13:49] [V] [TRT] Shape_581 [Shape] inputs: [onnx::Shape_757 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_581 for ONNX node: Shape_581 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_761 for ONNX tensor: onnx::Gather_761 [05/12/2023-15:13:49] [V] [TRT] Shape_581 [Shape] outputs: [onnx::Gather_761 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_582 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_582 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_582 [Constant] outputs: [onnx::Gather_762 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_583 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_761 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_762 [05/12/2023-15:13:49] [V] [TRT] Gather_583 [Gather] inputs: [onnx::Gather_761 -> (3)[INT32]], [onnx::Gather_762 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_762 for ONNX node: onnx::Gather_762 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_583 for ONNX node: Gather_583 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_763 for ONNX tensor: onnx::Unsqueeze_763 [05/12/2023-15:13:49] [V] [TRT] Gather_583 [Gather] outputs: [onnx::Unsqueeze_763 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_584 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_760 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_584 [Unsqueeze] inputs: [onnx::Unsqueeze_760 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_584 for ONNX node: Unsqueeze_584 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_766 for ONNX tensor: onnx::Concat_766 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_584 [Unsqueeze] outputs: [onnx::Concat_766 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_585 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_763 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_585 [Unsqueeze] inputs: [onnx::Unsqueeze_763 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_585 for ONNX node: Unsqueeze_585 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_767 for ONNX tensor: onnx::Concat_767 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_585 [Unsqueeze] outputs: [onnx::Concat_767 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_586 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_586 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_586 [Constant] outputs: [onnx::Concat_927 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_587 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_587 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_587 [Constant] outputs: [onnx::Concat_928 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Concat_588 [Concat] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_766 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_767 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_927 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_928 [05/12/2023-15:13:49] [V] [TRT] Concat_588 [Concat] inputs: [onnx::Concat_766 -> (1)[INT32]], [onnx::Concat_767 -> (1)[INT32]], [onnx::Concat_927 -> (1)[INT32]], [onnx::Concat_928 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_927 for ONNX node: onnx::Concat_927 [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_928 for ONNX node: onnx::Concat_928 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Concat_588 for ONNX node: Concat_588 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Reshape_770 for ONNX tensor: onnx::Reshape_770 [05/12/2023-15:13:49] [V] [TRT] Concat_588 [Concat] outputs: [onnx::Reshape_770 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Reshape_589 [Reshape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_757 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Reshape_770 [05/12/2023-15:13:49] [V] [TRT] Reshape_589 [Reshape] inputs: [onnx::Shape_757 -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_770 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Reshape_589 for ONNX node: Reshape_589 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Transpose_771 for ONNX tensor: onnx::Transpose_771 [05/12/2023-15:13:49] [V] [TRT] Reshape_589 [Reshape] outputs: [onnx::Transpose_771 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Transpose_590 [Transpose] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Transpose_771 [05/12/2023-15:13:49] [V] [TRT] Transpose_590 [Transpose] inputs: [onnx::Transpose_771 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Transpose_590 for ONNX node: Transpose_590 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_772 for ONNX tensor: onnx::MatMul_772 [05/12/2023-15:13:49] [V] [TRT] Transpose_590 [Transpose] outputs: [onnx::MatMul_772 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_591 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: mixed_query_layer.19 [05/12/2023-15:13:49] [V] [TRT] Shape_591 [Shape] inputs: [mixed_query_layer.19 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_591 for ONNX node: Shape_591 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_773 for ONNX tensor: onnx::Gather_773 [05/12/2023-15:13:49] [V] [TRT] Shape_591 [Shape] outputs: [onnx::Gather_773 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_592 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_592 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_592 [Constant] outputs: [onnx::Gather_774 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_593 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_773 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_774 [05/12/2023-15:13:49] [V] [TRT] Gather_593 [Gather] inputs: [onnx::Gather_773 -> (3)[INT32]], [onnx::Gather_774 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_774 for ONNX node: onnx::Gather_774 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_593 for ONNX node: Gather_593 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_775 for ONNX tensor: onnx::Unsqueeze_775 [05/12/2023-15:13:49] [V] [TRT] Gather_593 [Gather] outputs: [onnx::Unsqueeze_775 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_594 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: mixed_query_layer.19 [05/12/2023-15:13:49] [V] [TRT] Shape_594 [Shape] inputs: [mixed_query_layer.19 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_594 for ONNX node: Shape_594 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_776 for ONNX tensor: onnx::Gather_776 [05/12/2023-15:13:49] [V] [TRT] Shape_594 [Shape] outputs: [onnx::Gather_776 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_595 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_595 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_595 [Constant] outputs: [onnx::Gather_777 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_596 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_776 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_777 [05/12/2023-15:13:49] [V] [TRT] Gather_596 [Gather] inputs: [onnx::Gather_776 -> (3)[INT32]], [onnx::Gather_777 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_777 for ONNX node: onnx::Gather_777 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_596 for ONNX node: Gather_596 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_778 for ONNX tensor: onnx::Unsqueeze_778 [05/12/2023-15:13:49] [V] [TRT] Gather_596 [Gather] outputs: [onnx::Unsqueeze_778 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_597 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_775 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_597 [Unsqueeze] inputs: [onnx::Unsqueeze_775 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_597 for ONNX node: Unsqueeze_597 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_781 for ONNX tensor: onnx::Concat_781 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_597 [Unsqueeze] outputs: [onnx::Concat_781 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_598 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_778 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_598 [Unsqueeze] inputs: [onnx::Unsqueeze_778 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_598 for ONNX node: Unsqueeze_598 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_782 for ONNX tensor: onnx::Concat_782 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_598 [Unsqueeze] outputs: [onnx::Concat_782 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_599 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_599 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_599 [Constant] outputs: [onnx::Concat_929 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_600 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_600 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_600 [Constant] outputs: [onnx::Concat_930 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Concat_601 [Concat] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_781 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_782 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_929 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_930 [05/12/2023-15:13:49] [V] [TRT] Concat_601 [Concat] inputs: [onnx::Concat_781 -> (1)[INT32]], [onnx::Concat_782 -> (1)[INT32]], [onnx::Concat_929 -> (1)[INT32]], [onnx::Concat_930 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_929 for ONNX node: onnx::Concat_929 [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_930 for ONNX node: onnx::Concat_930 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Concat_601 for ONNX node: Concat_601 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Reshape_785 for ONNX tensor: onnx::Reshape_785 [05/12/2023-15:13:49] [V] [TRT] Concat_601 [Concat] outputs: [onnx::Reshape_785 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Reshape_602 [Reshape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: mixed_query_layer.19 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Reshape_785 [05/12/2023-15:13:49] [V] [TRT] Reshape_602 [Reshape] inputs: [mixed_query_layer.19 -> (-1, -1, 384)[FLOAT]], [onnx::Reshape_785 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Reshape_602 for ONNX node: Reshape_602 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Transpose_786 for ONNX tensor: onnx::Transpose_786 [05/12/2023-15:13:49] [V] [TRT] Reshape_602 [Reshape] outputs: [onnx::Transpose_786 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Transpose_603 [Transpose] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Transpose_786 [05/12/2023-15:13:49] [V] [TRT] Transpose_603 [Transpose] inputs: [onnx::Transpose_786 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Transpose_603 for ONNX node: Transpose_603 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_787 for ONNX tensor: onnx::MatMul_787 [05/12/2023-15:13:49] [V] [TRT] Transpose_603 [Transpose] outputs: [onnx::MatMul_787 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Transpose_604 [Transpose] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Transpose_754 [05/12/2023-15:13:49] [V] [TRT] Transpose_604 [Transpose] inputs: [onnx::Transpose_754 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Transpose_604 for ONNX node: Transpose_604 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_788 for ONNX tensor: onnx::MatMul_788 [05/12/2023-15:13:49] [V] [TRT] Transpose_604 [Transpose] outputs: [onnx::MatMul_788 -> (-1, 12, 32, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_605 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_787 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_788 [05/12/2023-15:13:49] [V] [TRT] MatMul_605 [MatMul] inputs: [onnx::MatMul_787 -> (-1, 12, -1, 32)[FLOAT]], [onnx::MatMul_788 -> (-1, 12, 32, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_605 for ONNX node: MatMul_605 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Div_789 for ONNX tensor: onnx::Div_789 [05/12/2023-15:13:49] [V] [TRT] MatMul_605 [MatMul] outputs: [onnx::Div_789 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_606 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_606 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_606 [Constant] outputs: [onnx::Div_790 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Div_607 [Div] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_789 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_790 [05/12/2023-15:13:49] [V] [TRT] Div_607 [Div] inputs: [onnx::Div_789 -> (-1, 12, -1, -1)[FLOAT]], [onnx::Div_790 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Div_790 for ONNX node: onnx::Div_790 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Div_607 for ONNX node: Div_607 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_791 for ONNX tensor: onnx::Add_791 [05/12/2023-15:13:49] [V] [TRT] Div_607 [Div] outputs: [onnx::Add_791 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_608 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_791 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_138 [05/12/2023-15:13:49] [V] [TRT] Add_608 [Add] inputs: [onnx::Add_791 -> (-1, 12, -1, -1)[FLOAT]], [onnx::Add_138 -> (-1, 1, 1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_608 for ONNX node: Add_608 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.132 for ONNX tensor: input.132 [05/12/2023-15:13:49] [V] [TRT] Add_608 [Add] outputs: [input.132 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Softmax_609 [Softmax] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.132 [05/12/2023-15:13:49] [V] [TRT] Softmax_609 [Softmax] inputs: [input.132 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Softmax_609 for ONNX node: Softmax_609 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.136 for ONNX tensor: input.136 [05/12/2023-15:13:49] [V] [TRT] Softmax_609 [Softmax] outputs: [input.136 -> (-1, 12, -1, -1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_610 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.136 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_772 [05/12/2023-15:13:49] [V] [TRT] MatMul_610 [MatMul] inputs: [input.136 -> (-1, 12, -1, -1)[FLOAT]], [onnx::MatMul_772 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_610 for ONNX node: MatMul_610 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Transpose_794 for ONNX tensor: onnx::Transpose_794 [05/12/2023-15:13:49] [V] [TRT] MatMul_610 [MatMul] outputs: [onnx::Transpose_794 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Transpose_611 [Transpose] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Transpose_794 [05/12/2023-15:13:49] [V] [TRT] Transpose_611 [Transpose] inputs: [onnx::Transpose_794 -> (-1, 12, -1, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Transpose_611 for ONNX node: Transpose_611 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Shape_795 for ONNX tensor: onnx::Shape_795 [05/12/2023-15:13:49] [V] [TRT] Transpose_611 [Transpose] outputs: [onnx::Shape_795 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_612 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_795 [05/12/2023-15:13:49] [V] [TRT] Shape_612 [Shape] inputs: [onnx::Shape_795 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_612 for ONNX node: Shape_612 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_796 for ONNX tensor: onnx::Gather_796 [05/12/2023-15:13:49] [V] [TRT] Shape_612 [Shape] outputs: [onnx::Gather_796 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_613 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_613 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_613 [Constant] outputs: [onnx::Gather_797 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_614 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_796 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_797 [05/12/2023-15:13:49] [V] [TRT] Gather_614 [Gather] inputs: [onnx::Gather_796 -> (4)[INT32]], [onnx::Gather_797 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_797 for ONNX node: onnx::Gather_797 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_614 for ONNX node: Gather_614 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_798 for ONNX tensor: onnx::Unsqueeze_798 [05/12/2023-15:13:49] [V] [TRT] Gather_614 [Gather] outputs: [onnx::Unsqueeze_798 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Shape_615 [Shape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_795 [05/12/2023-15:13:49] [V] [TRT] Shape_615 [Shape] inputs: [onnx::Shape_795 -> (-1, -1, 12, 32)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Shape_615 for ONNX node: Shape_615 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gather_799 for ONNX tensor: onnx::Gather_799 [05/12/2023-15:13:49] [V] [TRT] Shape_615 [Shape] outputs: [onnx::Gather_799 -> (4)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_616 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_616 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_616 [Constant] outputs: [onnx::Gather_800 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_617 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_799 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_800 [05/12/2023-15:13:49] [V] [TRT] Gather_617 [Gather] inputs: [onnx::Gather_799 -> (4)[INT32]], [onnx::Gather_800 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_800 for ONNX node: onnx::Gather_800 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 0 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_617 for ONNX node: Gather_617 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Unsqueeze_801 for ONNX tensor: onnx::Unsqueeze_801 [05/12/2023-15:13:49] [V] [TRT] Gather_617 [Gather] outputs: [onnx::Unsqueeze_801 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_618 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_798 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_618 [Unsqueeze] inputs: [onnx::Unsqueeze_798 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_618 for ONNX node: Unsqueeze_618 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_803 for ONNX tensor: onnx::Concat_803 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_618 [Unsqueeze] outputs: [onnx::Concat_803 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Unsqueeze_619 [Unsqueeze] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Unsqueeze_801 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_619 [Unsqueeze] inputs: [onnx::Unsqueeze_801 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Original shape: (), unsqueezing to: (1,) [05/12/2023-15:13:49] [V] [TRT] Registering layer: Unsqueeze_619 for ONNX node: Unsqueeze_619 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Concat_804 for ONNX tensor: onnx::Concat_804 [05/12/2023-15:13:49] [V] [TRT] Unsqueeze_619 [Unsqueeze] outputs: [onnx::Concat_804 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_620 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_620 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_620 [Constant] outputs: [onnx::Concat_931 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Concat_621 [Concat] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_803 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_804 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Concat_931 [05/12/2023-15:13:49] [V] [TRT] Concat_621 [Concat] inputs: [onnx::Concat_803 -> (1)[INT32]], [onnx::Concat_804 -> (1)[INT32]], [onnx::Concat_931 -> (1)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Concat_931 for ONNX node: onnx::Concat_931 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Concat_621 for ONNX node: Concat_621 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Reshape_806 for ONNX tensor: onnx::Reshape_806 [05/12/2023-15:13:49] [V] [TRT] Concat_621 [Concat] outputs: [onnx::Reshape_806 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Reshape_622 [Reshape] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Shape_795 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Reshape_806 [05/12/2023-15:13:49] [V] [TRT] Reshape_622 [Reshape] inputs: [onnx::Shape_795 -> (-1, -1, 12, 32)[FLOAT]], [onnx::Reshape_806 -> (3)[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Reshape_622 for ONNX node: Reshape_622 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_807 for ONNX tensor: onnx::MatMul_807 [05/12/2023-15:13:49] [V] [TRT] Reshape_622 [Reshape] outputs: [onnx::MatMul_807 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_623 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_807 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_932 [05/12/2023-15:13:49] [V] [TRT] MatMul_623 [MatMul] inputs: [onnx::MatMul_807 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_932 -> (384, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_932 for ONNX node: onnx::MatMul_932 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_623 for ONNX node: MatMul_623 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_809 for ONNX tensor: onnx::Add_809 [05/12/2023-15:13:49] [V] [TRT] MatMul_623 [MatMul] outputs: [onnx::Add_809 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_624 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.5.attention.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_809 [05/12/2023-15:13:49] [V] [TRT] Add_624 [Add] inputs: [encoder.layer.5.attention.output.dense.bias -> (384)[FLOAT]], [onnx::Add_809 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.5.attention.output.dense.bias for ONNX node: encoder.layer.5.attention.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_624 for ONNX node: Add_624 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.140 for ONNX tensor: input.140 [05/12/2023-15:13:49] [V] [TRT] Add_624 [Add] outputs: [input.140 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_625 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.140 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_734 [05/12/2023-15:13:49] [V] [TRT] Add_625 [Add] inputs: [input.140 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_734 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_625 for ONNX node: Add_625 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.144 for ONNX tensor: input.144 [05/12/2023-15:13:49] [V] [TRT] Add_625 [Add] outputs: [input.144 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: ReduceMean_626 [ReduceMean] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.144 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_626 [ReduceMean] inputs: [input.144 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: ReduceMean_626 for ONNX node: ReduceMean_626 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Sub_812 for ONNX tensor: onnx::Sub_812 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_626 [ReduceMean] outputs: [onnx::Sub_812 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Sub_627 [Sub] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.144 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Sub_812 [05/12/2023-15:13:49] [V] [TRT] Sub_627 [Sub] inputs: [input.144 -> (-1, -1, 384)[FLOAT]], [onnx::Sub_812 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Sub_627 for ONNX node: Sub_627 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Pow_813 for ONNX tensor: onnx::Pow_813 [05/12/2023-15:13:49] [V] [TRT] Sub_627 [Sub] outputs: [onnx::Pow_813 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_628 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_628 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_628 [Constant] outputs: [onnx::Pow_814 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Pow_629 [Pow] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_813 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_814 [05/12/2023-15:13:49] [V] [TRT] Pow_629 [Pow] inputs: [onnx::Pow_813 -> (-1, -1, 384)[FLOAT]], [onnx::Pow_814 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Pow_814 for ONNX node: onnx::Pow_814 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Pow_629 for ONNX node: Pow_629 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::ReduceMean_815 for ONNX tensor: onnx::ReduceMean_815 [05/12/2023-15:13:49] [V] [TRT] Pow_629 [Pow] outputs: [onnx::ReduceMean_815 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: ReduceMean_630 [ReduceMean] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::ReduceMean_815 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_630 [ReduceMean] inputs: [onnx::ReduceMean_815 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: ReduceMean_630 for ONNX node: ReduceMean_630 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_816 for ONNX tensor: onnx::Add_816 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_630 [ReduceMean] outputs: [onnx::Add_816 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_631 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_631 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_631 [Constant] outputs: [onnx::Add_817 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_632 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_816 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_817 [05/12/2023-15:13:49] [V] [TRT] Add_632 [Add] inputs: [onnx::Add_816 -> (-1, -1, 1)[FLOAT]], [onnx::Add_817 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Add_817 for ONNX node: onnx::Add_817 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_632 for ONNX node: Add_632 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Sqrt_818 for ONNX tensor: onnx::Sqrt_818 [05/12/2023-15:13:49] [V] [TRT] Add_632 [Add] outputs: [onnx::Sqrt_818 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Sqrt_633 [Sqrt] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Sqrt_818 [05/12/2023-15:13:49] [V] [TRT] Sqrt_633 [Sqrt] inputs: [onnx::Sqrt_818 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Sqrt_633 for ONNX node: Sqrt_633 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Div_819 for ONNX tensor: onnx::Div_819 [05/12/2023-15:13:49] [V] [TRT] Sqrt_633 [Sqrt] outputs: [onnx::Div_819 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Div_634 [Div] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_813 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_819 [05/12/2023-15:13:49] [V] [TRT] Div_634 [Div] inputs: [onnx::Pow_813 -> (-1, -1, 384)[FLOAT]], [onnx::Div_819 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Div_634 for ONNX node: Div_634 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Mul_820 for ONNX tensor: onnx::Mul_820 [05/12/2023-15:13:49] [V] [TRT] Div_634 [Div] outputs: [onnx::Mul_820 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Mul_635 [Mul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_820 [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.5.attention.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] Mul_635 [Mul] inputs: [onnx::Mul_820 -> (-1, -1, 384)[FLOAT]], [encoder.layer.5.attention.output.LayerNorm.weight -> (384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.5.attention.output.LayerNorm.weight for ONNX node: encoder.layer.5.attention.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] Registering layer: Mul_635 for ONNX node: Mul_635 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_821 for ONNX tensor: onnx::Add_821 [05/12/2023-15:13:49] [V] [TRT] Mul_635 [Mul] outputs: [onnx::Add_821 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_636 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_821 [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.5.attention.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] Add_636 [Add] inputs: [onnx::Add_821 -> (-1, -1, 384)[FLOAT]], [encoder.layer.5.attention.output.LayerNorm.bias -> (384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.5.attention.output.LayerNorm.bias for ONNX node: encoder.layer.5.attention.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_636 for ONNX node: Add_636 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: attention_output.19 for ONNX tensor: attention_output.19 [05/12/2023-15:13:49] [V] [TRT] Add_636 [Add] outputs: [attention_output.19 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_637 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: attention_output.19 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_933 [05/12/2023-15:13:49] [V] [TRT] MatMul_637 [MatMul] inputs: [attention_output.19 -> (-1, -1, 384)[FLOAT]], [onnx::MatMul_933 -> (384, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_933 for ONNX node: onnx::MatMul_933 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_637 for ONNX node: MatMul_637 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_824 for ONNX tensor: onnx::Add_824 [05/12/2023-15:13:49] [V] [TRT] MatMul_637 [MatMul] outputs: [onnx::Add_824 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_638 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.5.intermediate.dense.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_824 [05/12/2023-15:13:49] [V] [TRT] Add_638 [Add] inputs: [encoder.layer.5.intermediate.dense.bias -> (1536)[FLOAT]], [onnx::Add_824 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.5.intermediate.dense.bias for ONNX node: encoder.layer.5.intermediate.dense.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_638 for ONNX node: Add_638 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Div_825 for ONNX tensor: onnx::Div_825 [05/12/2023-15:13:49] [V] [TRT] Add_638 [Add] outputs: [onnx::Div_825 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_639 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_639 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_639 [Constant] outputs: [onnx::Div_826 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Div_640 [Div] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_825 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_826 [05/12/2023-15:13:49] [V] [TRT] Div_640 [Div] inputs: [onnx::Div_825 -> (-1, -1, 1536)[FLOAT]], [onnx::Div_826 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Div_826 for ONNX node: onnx::Div_826 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Div_640 for ONNX node: Div_640 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Erf_827 for ONNX tensor: onnx::Erf_827 [05/12/2023-15:13:49] [V] [TRT] Div_640 [Div] outputs: [onnx::Erf_827 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Erf_641 [Erf] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Erf_827 [05/12/2023-15:13:49] [V] [TRT] Erf_641 [Erf] inputs: [onnx::Erf_827 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Erf_641 for ONNX node: Erf_641 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_828 for ONNX tensor: onnx::Add_828 [05/12/2023-15:13:49] [V] [TRT] Erf_641 [Erf] outputs: [onnx::Add_828 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_642 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_642 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_642 [Constant] outputs: [onnx::Add_829 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_643 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_828 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_829 [05/12/2023-15:13:49] [V] [TRT] Add_643 [Add] inputs: [onnx::Add_828 -> (-1, -1, 1536)[FLOAT]], [onnx::Add_829 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Add_829 for ONNX node: onnx::Add_829 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_643 for ONNX node: Add_643 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Mul_830 for ONNX tensor: onnx::Mul_830 [05/12/2023-15:13:49] [V] [TRT] Add_643 [Add] outputs: [onnx::Mul_830 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Mul_644 [Mul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_825 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_830 [05/12/2023-15:13:49] [V] [TRT] Mul_644 [Mul] inputs: [onnx::Div_825 -> (-1, -1, 1536)[FLOAT]], [onnx::Mul_830 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Mul_644 for ONNX node: Mul_644 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Mul_831 for ONNX tensor: onnx::Mul_831 [05/12/2023-15:13:49] [V] [TRT] Mul_644 [Mul] outputs: [onnx::Mul_831 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_645 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_645 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_645 [Constant] outputs: [onnx::Mul_832 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Mul_646 [Mul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_831 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_832 [05/12/2023-15:13:49] [V] [TRT] Mul_646 [Mul] inputs: [onnx::Mul_831 -> (-1, -1, 1536)[FLOAT]], [onnx::Mul_832 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Mul_832 for ONNX node: onnx::Mul_832 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Mul_646 for ONNX node: Mul_646 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::MatMul_833 for ONNX tensor: onnx::MatMul_833 [05/12/2023-15:13:49] [V] [TRT] Mul_646 [Mul] outputs: [onnx::MatMul_833 -> (-1, -1, 1536)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: MatMul_647 [MatMul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_833 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::MatMul_934 [05/12/2023-15:13:49] [V] [TRT] MatMul_647 [MatMul] inputs: [onnx::MatMul_833 -> (-1, -1, 1536)[FLOAT]], [onnx::MatMul_934 -> (1536, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::MatMul_934 for ONNX node: onnx::MatMul_934 [05/12/2023-15:13:49] [V] [TRT] Registering layer: MatMul_647 for ONNX node: MatMul_647 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_835 for ONNX tensor: onnx::Add_835 [05/12/2023-15:13:49] [V] [TRT] MatMul_647 [MatMul] outputs: [onnx::Add_835 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_648 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.5.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_835 [05/12/2023-15:13:49] [V] [TRT] Add_648 [Add] inputs: [encoder.layer.5.output.dense.bias -> (384)[FLOAT]], [onnx::Add_835 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.5.output.dense.bias for ONNX node: encoder.layer.5.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_648 for ONNX node: Add_648 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.148 for ONNX tensor: input.148 [05/12/2023-15:13:49] [V] [TRT] Add_648 [Add] outputs: [input.148 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_649 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.148 [05/12/2023-15:13:49] [V] [TRT] Searching for input: attention_output.19 [05/12/2023-15:13:49] [V] [TRT] Add_649 [Add] inputs: [input.148 -> (-1, -1, 384)[FLOAT]], [attention_output.19 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_649 for ONNX node: Add_649 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: input.152 for ONNX tensor: input.152 [05/12/2023-15:13:49] [V] [TRT] Add_649 [Add] outputs: [input.152 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: ReduceMean_650 [ReduceMean] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.152 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_650 [ReduceMean] inputs: [input.152 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: ReduceMean_650 for ONNX node: ReduceMean_650 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Sub_838 for ONNX tensor: onnx::Sub_838 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_650 [ReduceMean] outputs: [onnx::Sub_838 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Sub_651 [Sub] [05/12/2023-15:13:49] [V] [TRT] Searching for input: input.152 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Sub_838 [05/12/2023-15:13:49] [V] [TRT] Sub_651 [Sub] inputs: [input.152 -> (-1, -1, 384)[FLOAT]], [onnx::Sub_838 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Sub_651 for ONNX node: Sub_651 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Pow_839 for ONNX tensor: onnx::Pow_839 [05/12/2023-15:13:49] [V] [TRT] Sub_651 [Sub] outputs: [onnx::Pow_839 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_652 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_652 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_652 [Constant] outputs: [onnx::Pow_840 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Pow_653 [Pow] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_839 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_840 [05/12/2023-15:13:49] [V] [TRT] Pow_653 [Pow] inputs: [onnx::Pow_839 -> (-1, -1, 384)[FLOAT]], [onnx::Pow_840 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Pow_840 for ONNX node: onnx::Pow_840 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Pow_653 for ONNX node: Pow_653 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::ReduceMean_841 for ONNX tensor: onnx::ReduceMean_841 [05/12/2023-15:13:49] [V] [TRT] Pow_653 [Pow] outputs: [onnx::ReduceMean_841 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: ReduceMean_654 [ReduceMean] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::ReduceMean_841 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_654 [ReduceMean] inputs: [onnx::ReduceMean_841 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: ReduceMean_654 for ONNX node: ReduceMean_654 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_842 for ONNX tensor: onnx::Add_842 [05/12/2023-15:13:49] [V] [TRT] ReduceMean_654 [ReduceMean] outputs: [onnx::Add_842 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_655 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_655 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_655 [Constant] outputs: [onnx::Add_843 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_656 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_842 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_843 [05/12/2023-15:13:49] [V] [TRT] Add_656 [Add] inputs: [onnx::Add_842 -> (-1, -1, 1)[FLOAT]], [onnx::Add_843 -> ()[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Add_843 for ONNX node: onnx::Add_843 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_656 for ONNX node: Add_656 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Sqrt_844 for ONNX tensor: onnx::Sqrt_844 [05/12/2023-15:13:49] [V] [TRT] Add_656 [Add] outputs: [onnx::Sqrt_844 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Sqrt_657 [Sqrt] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Sqrt_844 [05/12/2023-15:13:49] [V] [TRT] Sqrt_657 [Sqrt] inputs: [onnx::Sqrt_844 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Sqrt_657 for ONNX node: Sqrt_657 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Div_845 for ONNX tensor: onnx::Div_845 [05/12/2023-15:13:49] [V] [TRT] Sqrt_657 [Sqrt] outputs: [onnx::Div_845 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Div_658 [Div] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Pow_839 [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Div_845 [05/12/2023-15:13:49] [V] [TRT] Div_658 [Div] inputs: [onnx::Pow_839 -> (-1, -1, 384)[FLOAT]], [onnx::Div_845 -> (-1, -1, 1)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Div_658 for ONNX node: Div_658 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Mul_846 for ONNX tensor: onnx::Mul_846 [05/12/2023-15:13:49] [V] [TRT] Div_658 [Div] outputs: [onnx::Mul_846 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Mul_659 [Mul] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Mul_846 [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.5.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] Mul_659 [Mul] inputs: [onnx::Mul_846 -> (-1, -1, 384)[FLOAT]], [encoder.layer.5.output.LayerNorm.weight -> (384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.5.output.LayerNorm.weight for ONNX node: encoder.layer.5.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] Registering layer: Mul_659 for ONNX node: Mul_659 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Add_847 for ONNX tensor: onnx::Add_847 [05/12/2023-15:13:49] [V] [TRT] Mul_659 [Mul] outputs: [onnx::Add_847 -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Add_660 [Add] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Add_847 [05/12/2023-15:13:49] [V] [TRT] Searching for input: encoder.layer.5.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] Add_660 [Add] inputs: [onnx::Add_847 -> (-1, -1, 384)[FLOAT]], [encoder.layer.5.output.LayerNorm.bias -> (384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: encoder.layer.5.output.LayerNorm.bias for ONNX node: encoder.layer.5.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] Registering layer: Add_660 for ONNX node: Add_660 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: output_147 for ONNX tensor: output [05/12/2023-15:13:49] [V] [TRT] Add_660 [Add] outputs: [output -> (-1, -1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Constant_661 [Constant] [05/12/2023-15:13:49] [V] [TRT] Constant_661 [Constant] inputs: [05/12/2023-15:13:49] [V] [TRT] Constant_661 [Constant] outputs: [onnx::Gather_849 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gather_662 [Gather] [05/12/2023-15:13:49] [V] [TRT] Searching for input: output [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gather_849 [05/12/2023-15:13:49] [V] [TRT] Gather_662 [Gather] inputs: [output -> (-1, -1, 384)[FLOAT]], [onnx::Gather_849 -> ()[INT32]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: onnx::Gather_849 for ONNX node: onnx::Gather_849 [05/12/2023-15:13:49] [V] [TRT] Using Gather axis: 1 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gather_662 for ONNX node: Gather_662 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Gemm_850 for ONNX tensor: onnx::Gemm_850 [05/12/2023-15:13:49] [V] [TRT] Gather_662 [Gather] outputs: [onnx::Gemm_850 -> (-1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Gemm_663 [Gemm] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Gemm_850 [05/12/2023-15:13:49] [V] [TRT] Searching for input: pooler.dense.weight [05/12/2023-15:13:49] [V] [TRT] Searching for input: pooler.dense.bias [05/12/2023-15:13:49] [V] [TRT] Gemm_663 [Gemm] inputs: [onnx::Gemm_850 -> (-1, 384)[FLOAT]], [pooler.dense.weight -> (384, 384)[FLOAT]], [pooler.dense.bias -> (384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: pooler.dense.weight for ONNX node: pooler.dense.weight [05/12/2023-15:13:49] [V] [TRT] Using opA: 0 opB: 1 [05/12/2023-15:13:49] [V] [TRT] Registering layer: Gemm_663 for ONNX node: Gemm_663 [05/12/2023-15:13:49] [V] [TRT] Registering layer: pooler.dense.bias for ONNX node: pooler.dense.bias [05/12/2023-15:13:49] [V] [TRT] Registering tensor: onnx::Tanh_851 for ONNX tensor: onnx::Tanh_851 [05/12/2023-15:13:49] [V] [TRT] Gemm_663 [Gemm] outputs: [onnx::Tanh_851 -> (-1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Parsing node: Tanh_664 [Tanh] [05/12/2023-15:13:49] [V] [TRT] Searching for input: onnx::Tanh_851 [05/12/2023-15:13:49] [V] [TRT] Tanh_664 [Tanh] inputs: [onnx::Tanh_851 -> (-1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Registering layer: Tanh_664 for ONNX node: Tanh_664 [05/12/2023-15:13:49] [V] [TRT] Registering tensor: pooled_embeds_149 for ONNX tensor: pooled_embeds [05/12/2023-15:13:49] [V] [TRT] Tanh_664 [Tanh] outputs: [pooled_embeds -> (-1, 384)[FLOAT]], [05/12/2023-15:13:49] [V] [TRT] Marking output_147 as output: output [05/12/2023-15:13:49] [V] [TRT] Marking pooled_embeds_149 as output: pooled_embeds [05/12/2023-15:13:49] [I] Finish parsing network model [05/12/2023-15:13:49] [V] [TRT] Original: 647 layers [05/12/2023-15:13:49] [V] [TRT] After dead-layer removal: 647 layers [05/12/2023-15:13:49] [V] [TRT] Applying generic optimizations to the graph for inference. [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Sub_135 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Sub_135 with (Unnamed Layer* 62) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Mul_137 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Mul_137 with (Unnamed Layer* 65) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Pow_156 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Pow_156 with (Unnamed Layer* 99) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_159 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_159 with (Unnamed Layer* 103) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on embeddings.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing embeddings.LayerNorm.weight with (Unnamed Layer* 108) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on embeddings.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing embeddings.LayerNorm.bias with (Unnamed Layer* 111) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_857 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_857 with (Unnamed Layer* 114) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.0.attention.self.query.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.0.attention.self.query.bias with (Unnamed Layer* 117) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_858 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_858 with (Unnamed Layer* 120) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.0.attention.self.key.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.0.attention.self.key.bias with (Unnamed Layer* 123) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_861 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_861 with (Unnamed Layer* 138) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.0.attention.self.value.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.0.attention.self.value.bias with (Unnamed Layer* 141) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Div_220 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Div_220 with (Unnamed Layer* 172) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_867 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_867 with (Unnamed Layer* 204) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.0.attention.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.0.attention.output.dense.bias with (Unnamed Layer* 207) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Pow_244 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Pow_244 with (Unnamed Layer* 213) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_247 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_247 with (Unnamed Layer* 217) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.0.attention.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.0.attention.output.LayerNorm.weight with (Unnamed Layer* 222) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.0.attention.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.0.attention.output.LayerNorm.bias with (Unnamed Layer* 225) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_868 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_868 with (Unnamed Layer* 228) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.0.intermediate.dense.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.0.intermediate.dense.bias with (Unnamed Layer* 231) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Div_256 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Div_256 with (Unnamed Layer* 234) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_259 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_259 with (Unnamed Layer* 238) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Mul_262 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Mul_262 with (Unnamed Layer* 242) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_869 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_869 with (Unnamed Layer* 245) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.0.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.0.output.dense.bias with (Unnamed Layer* 248) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Pow_270 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Pow_270 with (Unnamed Layer* 254) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_273 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_273 with (Unnamed Layer* 258) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.0.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.0.output.LayerNorm.weight with (Unnamed Layer* 263) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.0.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.0.output.LayerNorm.bias with (Unnamed Layer* 266) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_870 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_870 with (Unnamed Layer* 269) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.1.attention.self.query.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.1.attention.self.query.bias with (Unnamed Layer* 272) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_871 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_871 with (Unnamed Layer* 275) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.1.attention.self.key.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.1.attention.self.key.bias with (Unnamed Layer* 278) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_874 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_874 with (Unnamed Layer* 293) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.1.attention.self.value.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.1.attention.self.value.bias with (Unnamed Layer* 296) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Div_334 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Div_334 with (Unnamed Layer* 327) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_880 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_880 with (Unnamed Layer* 359) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.1.attention.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.1.attention.output.dense.bias with (Unnamed Layer* 362) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Pow_358 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Pow_358 with (Unnamed Layer* 368) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_361 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_361 with (Unnamed Layer* 372) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.1.attention.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.1.attention.output.LayerNorm.weight with (Unnamed Layer* 377) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.1.attention.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.1.attention.output.LayerNorm.bias with (Unnamed Layer* 380) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_881 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_881 with (Unnamed Layer* 383) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.1.intermediate.dense.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.1.intermediate.dense.bias with (Unnamed Layer* 386) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Div_370 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Div_370 with (Unnamed Layer* 389) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_373 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_373 with (Unnamed Layer* 393) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Mul_376 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Mul_376 with (Unnamed Layer* 397) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_882 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_882 with (Unnamed Layer* 400) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.1.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.1.output.dense.bias with (Unnamed Layer* 403) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Pow_384 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Pow_384 with (Unnamed Layer* 409) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_387 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_387 with (Unnamed Layer* 413) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.1.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.1.output.LayerNorm.weight with (Unnamed Layer* 418) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.1.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.1.output.LayerNorm.bias with (Unnamed Layer* 421) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_883 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_883 with (Unnamed Layer* 424) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.2.attention.self.query.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.2.attention.self.query.bias with (Unnamed Layer* 427) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_884 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_884 with (Unnamed Layer* 430) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.2.attention.self.key.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.2.attention.self.key.bias with (Unnamed Layer* 433) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_887 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_887 with (Unnamed Layer* 448) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.2.attention.self.value.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.2.attention.self.value.bias with (Unnamed Layer* 451) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Div_448 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Div_448 with (Unnamed Layer* 482) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_893 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_893 with (Unnamed Layer* 514) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.2.attention.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.2.attention.output.dense.bias with (Unnamed Layer* 517) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Pow_472 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Pow_472 with (Unnamed Layer* 523) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_475 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_475 with (Unnamed Layer* 527) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.2.attention.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.2.attention.output.LayerNorm.weight with (Unnamed Layer* 532) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.2.attention.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.2.attention.output.LayerNorm.bias with (Unnamed Layer* 535) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_894 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_894 with (Unnamed Layer* 538) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.2.intermediate.dense.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.2.intermediate.dense.bias with (Unnamed Layer* 541) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Div_484 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Div_484 with (Unnamed Layer* 544) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_487 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_487 with (Unnamed Layer* 548) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Mul_490 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Mul_490 with (Unnamed Layer* 552) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_895 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_895 with (Unnamed Layer* 555) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.2.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.2.output.dense.bias with (Unnamed Layer* 558) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Pow_498 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Pow_498 with (Unnamed Layer* 564) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_501 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_501 with (Unnamed Layer* 568) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.2.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.2.output.LayerNorm.weight with (Unnamed Layer* 573) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.2.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.2.output.LayerNorm.bias with (Unnamed Layer* 576) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_896 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_896 with (Unnamed Layer* 579) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.3.attention.self.query.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.3.attention.self.query.bias with (Unnamed Layer* 582) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_897 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_897 with (Unnamed Layer* 585) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.3.attention.self.key.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.3.attention.self.key.bias with (Unnamed Layer* 588) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_900 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_900 with (Unnamed Layer* 603) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.3.attention.self.value.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.3.attention.self.value.bias with (Unnamed Layer* 606) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Div_562 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Div_562 with (Unnamed Layer* 637) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_906 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_906 with (Unnamed Layer* 669) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.3.attention.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.3.attention.output.dense.bias with (Unnamed Layer* 672) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Pow_586 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Pow_586 with (Unnamed Layer* 678) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_589 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_589 with (Unnamed Layer* 682) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.3.attention.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.3.attention.output.LayerNorm.weight with (Unnamed Layer* 687) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.3.attention.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.3.attention.output.LayerNorm.bias with (Unnamed Layer* 690) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_907 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_907 with (Unnamed Layer* 693) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.3.intermediate.dense.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.3.intermediate.dense.bias with (Unnamed Layer* 696) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Div_598 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Div_598 with (Unnamed Layer* 699) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_601 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_601 with (Unnamed Layer* 703) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Mul_604 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Mul_604 with (Unnamed Layer* 707) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_908 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_908 with (Unnamed Layer* 710) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.3.output.dense.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.3.output.dense.bias with (Unnamed Layer* 713) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Pow_612 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Pow_612 with (Unnamed Layer* 719) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_615 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_615 with (Unnamed Layer* 723) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.3.output.LayerNorm.weight [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.3.output.LayerNorm.weight with (Unnamed Layer* 728) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.3.output.LayerNorm.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.3.output.LayerNorm.bias with (Unnamed Layer* 731) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_909 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_909 with (Unnamed Layer* 734) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.4.attention.self.query.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.4.attention.self.query.bias with (Unnamed Layer* 737) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_910 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_910 with (Unnamed Layer* 740) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.4.attention.self.key.bias [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.4.attention.self.key.bias with (Unnamed Layer* 743) [Shuffle] [05/12/2023-15:13:49] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_913 [05/12/2023-15:13:49] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_913 with (Unnamed Layer* 758) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.4.attention.self.value.bias [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.4.attention.self.value.bias with (Unnamed Layer* 761) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::Div_676 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::Div_676 with (Unnamed Layer* 792) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_919 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_919 with (Unnamed Layer* 824) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.4.attention.output.dense.bias [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.4.attention.output.dense.bias with (Unnamed Layer* 827) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::Pow_700 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::Pow_700 with (Unnamed Layer* 833) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_703 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_703 with (Unnamed Layer* 837) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.4.attention.output.LayerNorm.weight [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.4.attention.output.LayerNorm.weight with (Unnamed Layer* 842) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.4.attention.output.LayerNorm.bias [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.4.attention.output.LayerNorm.bias with (Unnamed Layer* 845) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_920 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_920 with (Unnamed Layer* 848) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.4.intermediate.dense.bias [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.4.intermediate.dense.bias with (Unnamed Layer* 851) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::Div_712 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::Div_712 with (Unnamed Layer* 854) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_715 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_715 with (Unnamed Layer* 858) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::Mul_718 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::Mul_718 with (Unnamed Layer* 862) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_921 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_921 with (Unnamed Layer* 865) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.4.output.dense.bias [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.4.output.dense.bias with (Unnamed Layer* 868) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::Pow_726 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::Pow_726 with (Unnamed Layer* 874) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_729 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_729 with (Unnamed Layer* 878) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.4.output.LayerNorm.weight [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.4.output.LayerNorm.weight with (Unnamed Layer* 883) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.4.output.LayerNorm.bias [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.4.output.LayerNorm.bias with (Unnamed Layer* 886) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_922 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_922 with (Unnamed Layer* 889) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.5.attention.self.query.bias [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.5.attention.self.query.bias with (Unnamed Layer* 892) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_923 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_923 with (Unnamed Layer* 895) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.5.attention.self.key.bias [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.5.attention.self.key.bias with (Unnamed Layer* 898) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_926 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_926 with (Unnamed Layer* 913) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.5.attention.self.value.bias [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.5.attention.self.value.bias with (Unnamed Layer* 916) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::Div_790 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::Div_790 with (Unnamed Layer* 947) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Unsqueeze_24 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Unsqueeze_24 with Unsqueeze_25 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_97 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_97 with Transpose_98 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_70 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_70 with Transpose_99 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_84 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_84 with Transpose_85 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Transpose_106 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Transpose_106 with Reshape_117 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_198 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_198 with Transpose_199 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_171 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_171 with Transpose_200 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_185 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_185 with Transpose_186 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Transpose_207 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Transpose_207 with Reshape_218 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_299 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_299 with Transpose_300 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_272 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_272 with Transpose_301 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_286 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_286 with Transpose_287 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Transpose_308 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Transpose_308 with Reshape_319 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_400 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_400 with Transpose_401 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_373 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_373 with Transpose_402 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_387 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_387 with Transpose_388 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Transpose_409 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Transpose_409 with Reshape_420 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_501 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_501 with Transpose_502 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_474 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_474 with Transpose_503 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_488 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_488 with Transpose_489 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Transpose_510 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Transpose_510 with Reshape_521 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_602 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_602 with Transpose_603 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_575 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_575 with Transpose_604 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Reshape_589 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Reshape_589 with Transpose_590 [05/12/2023-15:13:50] [V] [TRT] Running: ShuffleShuffleFusion on Transpose_611 [05/12/2023-15:13:50] [V] [TRT] ShuffleShuffleFusion: Fusing Transpose_611 with Reshape_622 [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_932 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_932 with (Unnamed Layer* 979) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.5.attention.output.dense.bias [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.5.attention.output.dense.bias with (Unnamed Layer* 982) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::Pow_814 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::Pow_814 with (Unnamed Layer* 988) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_817 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_817 with (Unnamed Layer* 992) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.5.attention.output.LayerNorm.weight [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.5.attention.output.LayerNorm.weight with (Unnamed Layer* 997) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.5.attention.output.LayerNorm.bias [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.5.attention.output.LayerNorm.bias with (Unnamed Layer* 1000) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_933 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_933 with (Unnamed Layer* 1003) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.5.intermediate.dense.bias [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.5.intermediate.dense.bias with (Unnamed Layer* 1006) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::Div_826 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::Div_826 with (Unnamed Layer* 1009) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_829 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_829 with (Unnamed Layer* 1013) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::Mul_832 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::Mul_832 with (Unnamed Layer* 1017) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_934 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_934 with (Unnamed Layer* 1020) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.5.output.dense.bias [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.5.output.dense.bias with (Unnamed Layer* 1023) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::Pow_840 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::Pow_840 with (Unnamed Layer* 1029) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on onnx::Add_843 [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing onnx::Add_843 with (Unnamed Layer* 1033) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.5.output.LayerNorm.weight [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.5.output.LayerNorm.weight with (Unnamed Layer* 1038) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on encoder.layer.5.output.LayerNorm.bias [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing encoder.layer.5.output.LayerNorm.bias with (Unnamed Layer* 1041) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Running: ConstShuffleFusion on pooler.dense.bias [05/12/2023-15:13:50] [V] [TRT] ConstShuffleFusion: Fusing pooler.dense.bias with (Unnamed Layer* 1048) [Shuffle] [05/12/2023-15:13:50] [V] [TRT] Found MatMul_55 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_57 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_71 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_100 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found Softmax_104 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_105 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_156 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_158 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_172 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_201 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found Softmax_205 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_206 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_257 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_259 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_273 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_302 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found Softmax_306 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_307 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_358 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_360 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_374 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_403 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found Softmax_407 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_408 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_459 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_461 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_475 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_504 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found Softmax_508 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_509 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_560 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_562 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_576 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_605 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found Softmax_609 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found MatMul_610 to be part of self-attention pattern. [05/12/2023-15:13:50] [V] [TRT] Found and reassigned Myelin backends for Self-Attention nodes [05/12/2023-15:13:50] [V] [TRT] After Myelin optimization: 1 layers [05/12/2023-15:13:50] [V] [TRT] Applying ScaleNodes fusions. [05/12/2023-15:13:50] [V] [TRT] After scale fusion: 1 layers [05/12/2023-15:13:50] [V] [TRT] After dupe layer removal: 1 layers [05/12/2023-15:13:50] [V] [TRT] After final dead-layer removal: 1 layers [05/12/2023-15:13:50] [V] [TRT] After tensor merging: 1 layers [05/12/2023-15:13:50] [V] [TRT] After vertical fusions: 1 layers [05/12/2023-15:13:50] [V] [TRT] After dupe layer removal: 1 layers [05/12/2023-15:13:50] [V] [TRT] After final dead-layer removal: 1 layers [05/12/2023-15:13:50] [V] [TRT] After tensor merging: 1 layers [05/12/2023-15:13:50] [V] [TRT] After slice removal: 1 layers [05/12/2023-15:13:50] [V] [TRT] After concat removal: 1 layers [05/12/2023-15:13:50] [V] [TRT] Trying to split Reshape and strided tensor [05/12/2023-15:13:50] [V] [TRT] Graph construction and optimization completed in 0.342967 seconds. [05/12/2023-15:13:50] [V] [TRT] Using cublas as plugin tactic source [05/12/2023-15:13:50] [V] [TRT] Using cublasLt as core library tactic source [05/12/2023-15:13:50] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +6, GPU +12, now: CPU 343, GPU 670 (MiB) [05/12/2023-15:13:50] [V] [TRT] Using cuDNN as plugin tactic source [05/12/2023-15:13:50] [V] [TRT] Using cuDNN as core library tactic source [05/12/2023-15:13:50] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +1, GPU +10, now: CPU 344, GPU 680 (MiB) [05/12/2023-15:13:50] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored. [05/12/2023-15:13:50] [V] [TRT] Constructing optimization profile number 0 [1/1]. [05/12/2023-15:13:50] [V] [TRT] Reserving memory for host IO tensors. Host: 0 bytes [05/12/2023-15:13:50] [V] [TRT] =============== Computing reformatting costs [05/12/2023-15:13:50] [V] [TRT] =============== Computing reformatting costs [05/12/2023-15:13:50] [V] [TRT] =============== Computing reformatting costs [05/12/2023-15:13:50] [V] [TRT] =============== Computing reformatting costs [05/12/2023-15:13:50] [V] [TRT] =============== Computing reformatting costs [05/12/2023-15:13:50] [V] [TRT] =============== Computing reformatting costs [05/12/2023-15:13:50] [V] [TRT] =============== Computing reformatting costs [05/12/2023-15:13:50] [V] [TRT] =============== Computing reformatting costs [05/12/2023-15:13:50] [V] [TRT] =============== Computing reformatting costs [05/12/2023-15:13:50] [V] [TRT] =============== Computing reformatting costs [05/12/2023-15:13:50] [V] [TRT] =============== Computing costs for [05/12/2023-15:13:50] [V] [TRT] *************** Autotuning format combination: -> Int32(), Int32(), Int32() *************** [05/12/2023-15:13:50] [V] [TRT] --------------- Timing Runner: [HostToDeviceCopy 0] (ShapeHostToDevice) [05/12/2023-15:13:50] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00772235 [05/12/2023-15:13:50] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.00772235 [05/12/2023-15:13:50] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: ShapeHostToDevice Tactic: 0x0000000000000000 [05/12/2023-15:13:50] [V] [TRT] =============== Computing costs for [05/12/2023-15:13:50] [V] [TRT] *************** Autotuning format combination: Int32(128,1), Int32(128,1), Int32(), Int32(), Int32() -> Float(49152,384,1), Float(384,1) *************** [05/12/2023-15:13:50] [V] [TRT] --------------- Timing Runner: {ForeignNode[embeddings.token_type_embeddings.weight...Tanh_664]} (Myelin) [05/12/2023-15:13:54] [V] [TRT] (foreignNode) Set user's cuda kernel library [05/12/2023-15:13:54] [V] [TRT] (foreignNode) Set user's cuda kernel library [05/12/2023-15:13:54] [V] [TRT] (foreignNode) Pass fuse_conv_padding is currently skipped for dynamic shapes [05/12/2023-15:13:54] [V] [TRT] (foreignNode) Pass pad_conv_channel is currently skipped for dynamic shapes [05/12/2023-15:13:54] [V] [TRT] (foreignNode) Pass fuse_conv_padding is currently skipped for dynamic shapes [05/12/2023-15:13:54] [V] [TRT] (foreignNode) Pass pad_conv_channel is currently skipped for dynamic shapes [05/12/2023-15:13:58] [V] [TRT] (foreignNode) Set user's cuda kernel library [05/12/2023-15:13:58] [V] [TRT] (foreignNode) Set user's cuda kernel library [05/12/2023-15:13:58] [V] [TRT] (foreignNode) Pass fuse_conv_padding is currently skipped for dynamic shapes [05/12/2023-15:13:58] [V] [TRT] (foreignNode) Pass pad_conv_channel is currently skipped for dynamic shapes [05/12/2023-15:13:58] [V] [TRT] (foreignNode) Pass fuse_conv_padding is currently skipped for dynamic shapes [05/12/2023-15:13:58] [V] [TRT] (foreignNode) Pass pad_conv_channel is currently skipped for dynamic shapes [05/12/2023-15:14:02] [V] [TRT] (foreignNode) Set user's cuda kernel library [05/12/2023-15:14:02] [V] [TRT] (foreignNode) Set user's cuda kernel library [05/12/2023-15:14:02] [V] [TRT] (foreignNode) Pass fuse_conv_padding is currently skipped for dynamic shapes [05/12/2023-15:14:02] [V] [TRT] (foreignNode) Pass pad_conv_channel is currently skipped for dynamic shapes [05/12/2023-15:14:02] [V] [TRT] (foreignNode) Pass fuse_conv_padding is currently skipped for dynamic shapes [05/12/2023-15:14:02] [V] [TRT] (foreignNode) Pass pad_conv_channel is currently skipped for dynamic shapes [05/12/2023-15:14:06] [V] [TRT] (foreignNode) Set user's cuda kernel library [05/12/2023-15:14:06] [V] [TRT] (foreignNode) Set user's cuda kernel library [05/12/2023-15:14:06] [V] [TRT] (foreignNode) Pass fuse_conv_padding is currently skipped for dynamic shapes [05/12/2023-15:14:06] [V] [TRT] (foreignNode) Pass pad_conv_channel is currently skipped for dynamic shapes [05/12/2023-15:14:06] [V] [TRT] (foreignNode) Pass fuse_conv_padding is currently skipped for dynamic shapes [05/12/2023-15:14:06] [V] [TRT] (foreignNode) Pass pad_conv_channel is currently skipped for dynamic shapes [05/12/2023-15:14:10] [V] [TRT] (foreignNode) Set user's cuda kernel library [05/12/2023-15:14:10] [V] [TRT] (foreignNode) Set user's cuda kernel library [05/12/2023-15:14:10] [V] [TRT] (foreignNode) Pass fuse_conv_padding is currently skipped for dynamic shapes [05/12/2023-15:14:10] [V] [TRT] (foreignNode) Pass pad_conv_channel is currently skipped for dynamic shapes [05/12/2023-15:14:10] [V] [TRT] (foreignNode) Pass fuse_conv_padding is currently skipped for dynamic shapes [05/12/2023-15:14:10] [V] [TRT] (foreignNode) Pass pad_conv_channel is currently skipped for dynamic shapes [05/12/2023-15:14:13] [V] [TRT] (foreignNode) Set user's cuda kernel library [05/12/2023-15:14:13] [V] [TRT] (foreignNode) Set user's cuda kernel library [05/12/2023-15:14:13] [V] [TRT] (foreignNode) Pass fuse_conv_padding is currently skipped for dynamic shapes [05/12/2023-15:14:13] [V] [TRT] (foreignNode) Pass pad_conv_channel is currently skipped for dynamic shapes [05/12/2023-15:14:13] [V] [TRT] (foreignNode) Pass fuse_conv_padding is currently skipped for dynamic shapes [05/12/2023-15:14:13] [V] [TRT] (foreignNode) Pass pad_conv_channel is currently skipped for dynamic shapes [05/12/2023-15:14:15] [V] [TRT] (foreignNode) Set user's cuda kernel library [05/12/2023-15:14:15] [V] [TRT] (foreignNode) Set user's cuda kernel library [05/12/2023-15:14:15] [V] [TRT] (foreignNode) Pass fuse_conv_padding is currently skipped for dynamic shapes [05/12/2023-15:14:15] [V] [TRT] (foreignNode) Pass pad_conv_channel is currently skipped for dynamic shapes [05/12/2023-15:14:15] [V] [TRT] (foreignNode) Pass fuse_conv_padding is currently skipped for dynamic shapes [05/12/2023-15:14:15] [V] [TRT] (foreignNode) Pass pad_conv_channel is currently skipped for dynamic shapes [05/12/2023-15:14:17] [V] [TRT] Tactic: 0x0000000000000000 Time: 5.39941 [05/12/2023-15:14:17] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 5.39941 [05/12/2023-15:14:17] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Myelin Tactic: 0x0000000000000000 [05/12/2023-15:14:17] [V] [TRT] Formats and tactics selection completed in 26.9378 seconds. [05/12/2023-15:14:17] [V] [TRT] After reformat layers: 2 layers [05/12/2023-15:14:17] [V] [TRT] Pre-optimized block assignment. [05/12/2023-15:14:17] [V] [TRT] Block size 512 [05/12/2023-15:14:17] [V] [TRT] Block size 512 [05/12/2023-15:14:17] [V] [TRT] Block size 512 [05/12/2023-15:14:17] [V] [TRT] Block size 14680064000 [05/12/2023-15:14:17] [V] [TRT] Total Activation Memory: 14680065536 [05/12/2023-15:14:17] [I] [TRT] Detected 2 inputs and 2 output network tensors. [05/12/2023-15:14:18] [V] [TRT] Layer: [HostToDeviceCopy 0] Host Persistent: 12 Device Persistent: 0 Scratch Memory: 0 [05/12/2023-15:14:18] [V] [TRT] Layer: {ForeignNode[embeddings.token_type_embeddings.weight...Tanh_664]} Host Persistent: 168 Device Persistent: 0 Scratch Memory: 94420992 [05/12/2023-15:14:18] [I] [TRT] Total Host Persistent Memory: 192 [05/12/2023-15:14:18] [I] [TRT] Total Device Persistent Memory: 0 [05/12/2023-15:14:18] [I] [TRT] Total Scratch Memory: 94420992 [05/12/2023-15:14:18] [I] [TRT] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 0 MiB, GPU 183 MiB [05/12/2023-15:14:18] [I] [TRT] [BlockAssignment] Algorithm ShiftNTopDown took 0.079698ms to assign 4 blocks to 4 nodes requiring 94422528 bytes. [05/12/2023-15:14:18] [V] [TRT] Optimized block assignment. [05/12/2023-15:14:18] [V] [TRT] Block size 94420992 [05/12/2023-15:14:18] [V] [TRT] Block size 512 [05/12/2023-15:14:18] [V] [TRT] Block size 512 [05/12/2023-15:14:18] [V] [TRT] Block size 512 [05/12/2023-15:14:18] [I] [TRT] Total Activation Memory: 94422528 [05/12/2023-15:14:18] [V] [TRT] Disabling unused tactic source: CUDNN [05/12/2023-15:14:18] [V] [TRT] Disabling unused tactic source: CUBLAS, CUBLAS_LT [05/12/2023-15:14:18] [V] [TRT] Disabling unused tactic source: EDGE_MASK_CONVOLUTIONS [05/12/2023-15:14:18] [V] [TRT] Disabling unused tactic source: JIT_CONVOLUTIONS [05/12/2023-15:14:18] [V] [TRT] Engine generation completed in 28.633 seconds. [05/12/2023-15:14:18] [V] [TRT] Engine Layer Information: Layer(ShapeHostToDevice): [HostToDeviceCopy 0], Tactic: 0x0000000000000000, -> token_ids[implicit padding mask tensor 0] (Int32[]), attn_mask[implicit padding mask tensor 1] (Int32[]), input.16'[implicit padding mask tensor 2] (Int32[]) Layer(Myelin): {ForeignNode[embeddings.token_type_embeddings.weight...Tanh_664]}, Tactic: 0x0000000000000000, token_ids (Int32[-1,128]), attn_mask (Int32[-1,128]), token_ids[implicit padding mask tensor 0] (Int32[]), attn_mask[implicit padding mask tensor 1] (Int32[]), input.16'[implicit padding mask tensor 2] (Int32[]) -> output (Float[-1,128,384]), pooled_embeds (Float[-1,384]) [05/12/2023-15:14:18] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +0, GPU +128, now: CPU 0, GPU 128 (MiB) [05/12/2023-15:14:19] [I] Engine built in 33.7058 sec. [05/12/2023-15:14:19] [I] [TRT] Loaded engine size: 88 MiB [05/12/2023-15:14:19] [V] [TRT] Deserialization required 84250 microseconds. [05/12/2023-15:14:19] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +86, now: CPU 0, GPU 86 (MiB) [05/12/2023-15:14:19] [I] Engine deserialized in 0.0855796 sec. [05/12/2023-15:14:19] [V] [TRT] Total per-runner device persistent memory is 0 [05/12/2023-15:14:19] [V] [TRT] Total per-runner host persistent memory is 192 [05/12/2023-15:14:19] [V] [TRT] Allocated activation device memory of size 94422528 [05/12/2023-15:14:19] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +90, now: CPU 0, GPU 176 (MiB) [05/12/2023-15:14:19] [V] [TRT] CUDA lazy loading is enabled. [05/12/2023-15:14:19] [I] Setting persistentCacheLimit to 0 bytes. [05/12/2023-15:14:19] [V] Using enqueueV3. [05/12/2023-15:14:19] [I] Using random values for input token_ids [05/12/2023-15:14:19] [I] Created input binding for token_ids with dimensions 16x128 [05/12/2023-15:14:19] [I] Using random values for input attn_mask [05/12/2023-15:14:19] [I] Created input binding for attn_mask with dimensions 16x128 [05/12/2023-15:14:19] [I] Using random values for output output [05/12/2023-15:14:19] [I] Created output binding for output with dimensions 16x128x384 [05/12/2023-15:14:19] [I] Using random values for output pooled_embeds [05/12/2023-15:14:19] [I] Created output binding for pooled_embeds with dimensions 16x384 [05/12/2023-15:14:19] [I] Starting inference [05/12/2023-15:14:23] [I] Warmup completed 37 queries over 200 ms [05/12/2023-15:14:23] [I] Timing trace has 602 queries over 3.00709 s [05/12/2023-15:14:23] [I] [05/12/2023-15:14:23] [I] === Trace details === [05/12/2023-15:14:23] [I] Trace averages of 10 runs: [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 5.0817 ms - Host latency: 5.38286 ms (enqueue 5.12796 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 5.09205 ms - Host latency: 5.39247 ms (enqueue 5.13767 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.99794 ms - Host latency: 5.29922 ms (enqueue 5.04429 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.90199 ms - Host latency: 5.20255 ms (enqueue 4.94735 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.89124 ms - Host latency: 5.19307 ms (enqueue 4.9374 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.88007 ms - Host latency: 5.18154 ms (enqueue 4.9252 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.86637 ms - Host latency: 5.16719 ms (enqueue 4.9124 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.9064 ms - Host latency: 5.20762 ms (enqueue 4.95215 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.87209 ms - Host latency: 5.17324 ms (enqueue 4.91838 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.90158 ms - Host latency: 5.20229 ms (enqueue 4.94766 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.8767 ms - Host latency: 5.17805 ms (enqueue 4.92278 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.8729 ms - Host latency: 5.17401 ms (enqueue 4.91885 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.88469 ms - Host latency: 5.18563 ms (enqueue 4.93073 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.88213 ms - Host latency: 5.1831 ms (enqueue 4.92849 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.88007 ms - Host latency: 5.18118 ms (enqueue 4.92692 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.85867 ms - Host latency: 5.16132 ms (enqueue 4.90502 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.87037 ms - Host latency: 5.17114 ms (enqueue 4.91661 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.87319 ms - Host latency: 5.17451 ms (enqueue 4.9196 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.87312 ms - Host latency: 5.17416 ms (enqueue 4.91876 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.89817 ms - Host latency: 5.19921 ms (enqueue 4.9438 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.87638 ms - Host latency: 5.17784 ms (enqueue 4.92286 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.87855 ms - Host latency: 5.18015 ms (enqueue 4.92445 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.87117 ms - Host latency: 5.17186 ms (enqueue 4.91752 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.86306 ms - Host latency: 5.16447 ms (enqueue 4.90952 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.9067 ms - Host latency: 5.20757 ms (enqueue 4.95281 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.8599 ms - Host latency: 5.16085 ms (enqueue 4.90626 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.89036 ms - Host latency: 5.19117 ms (enqueue 4.93702 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.88765 ms - Host latency: 5.1885 ms (enqueue 4.93401 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.8652 ms - Host latency: 5.16604 ms (enqueue 4.91096 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.87965 ms - Host latency: 5.18064 ms (enqueue 4.92621 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.86143 ms - Host latency: 5.16232 ms (enqueue 4.90853 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.86327 ms - Host latency: 5.16415 ms (enqueue 4.90964 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.89962 ms - Host latency: 5.20131 ms (enqueue 4.94481 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.84149 ms - Host latency: 5.14385 ms (enqueue 4.88762 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.89214 ms - Host latency: 5.19265 ms (enqueue 4.93688 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.87395 ms - Host latency: 5.17433 ms (enqueue 4.91984 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.8766 ms - Host latency: 5.1811 ms (enqueue 4.92318 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.89275 ms - Host latency: 5.19436 ms (enqueue 4.93855 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.8814 ms - Host latency: 5.18262 ms (enqueue 4.92695 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.89177 ms - Host latency: 5.19304 ms (enqueue 4.93772 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.89612 ms - Host latency: 5.19727 ms (enqueue 4.9418 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.87192 ms - Host latency: 5.17354 ms (enqueue 4.91738 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.84312 ms - Host latency: 5.14399 ms (enqueue 4.88894 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.85674 ms - Host latency: 5.15842 ms (enqueue 4.90354 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.86184 ms - Host latency: 5.16313 ms (enqueue 4.90781 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.87773 ms - Host latency: 5.17903 ms (enqueue 4.92371 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.87886 ms - Host latency: 5.17991 ms (enqueue 4.92476 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.87524 ms - Host latency: 5.17695 ms (enqueue 4.922 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.86711 ms - Host latency: 5.16875 ms (enqueue 4.91416 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.88508 ms - Host latency: 5.1863 ms (enqueue 4.9314 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.88064 ms - Host latency: 5.18254 ms (enqueue 4.92681 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.86035 ms - Host latency: 5.16125 ms (enqueue 4.90605 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.86682 ms - Host latency: 5.1675 ms (enqueue 4.91196 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.8957 ms - Host latency: 5.19658 ms (enqueue 4.94097 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.88267 ms - Host latency: 5.18435 ms (enqueue 4.92803 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.8845 ms - Host latency: 5.18665 ms (enqueue 4.93013 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.88159 ms - Host latency: 5.18259 ms (enqueue 4.92734 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.87002 ms - Host latency: 5.17092 ms (enqueue 4.91531 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.86221 ms - Host latency: 5.16309 ms (enqueue 4.90745 ms) [05/12/2023-15:14:23] [I] Average on 10 runs - GPU latency: 4.86213 ms - Host latency: 5.16279 ms (enqueue 4.90681 ms) [05/12/2023-15:14:23] [I] [05/12/2023-15:14:23] [I] === Performance summary === [05/12/2023-15:14:23] [I] Throughput: 200.193 qps [05/12/2023-15:14:23] [I] Latency: min = 5.09311 ms, max = 5.48309 ms, mean = 5.18756 ms, median = 5.17706 ms, percentile(90%) = 5.23132 ms, percentile(95%) = 5.27832 ms, percentile(99%) = 5.40016 ms [05/12/2023-15:14:23] [I] Enqueue Time: min = 4.83563 ms, max = 5.22864 ms, mean = 4.93231 ms, median = 4.92169 ms, percentile(90%) = 4.97681 ms, percentile(95%) = 5.02307 ms, percentile(99%) = 5.14386 ms [05/12/2023-15:14:23] [I] H2D Latency: min = 0.0337219 ms, max = 0.0456543 ms, mean = 0.0366111 ms, median = 0.0366211 ms, percentile(90%) = 0.0371704 ms, percentile(95%) = 0.0373535 ms, percentile(99%) = 0.0383301 ms [05/12/2023-15:14:23] [I] GPU Compute Time: min = 4.79129 ms, max = 5.18042 ms, mean = 4.88633 ms, median = 4.87622 ms, percentile(90%) = 4.93152 ms, percentile(95%) = 4.97766 ms, percentile(99%) = 5.09747 ms [05/12/2023-15:14:23] [I] D2H Latency: min = 0.260986 ms, max = 0.28772 ms, mean = 0.264615 ms, median = 0.264282 ms, percentile(90%) = 0.265625 ms, percentile(95%) = 0.265869 ms, percentile(99%) = 0.269287 ms [05/12/2023-15:14:23] [I] Total Host Walltime: 3.00709 s [05/12/2023-15:14:23] [I] Total GPU Compute Time: 2.94157 s [05/12/2023-15:14:23] [I] Explanations of the performance metrics are printed in the verbose logs. [05/12/2023-15:14:23] [V] [05/12/2023-15:14:23] [V] === Explanations of the performance metrics === [05/12/2023-15:14:23] [V] Total Host Walltime: the host walltime from when the first query (after warmups) is enqueued to when the last query is completed. [05/12/2023-15:14:23] [V] GPU Compute Time: the GPU latency to execute the kernels for a query. [05/12/2023-15:14:23] [V] Total GPU Compute Time: the summation of the GPU Compute Time of all the queries. If this is significantly shorter than Total Host Walltime, the GPU may be under-utilized because of host-side overheads or data transfers. [05/12/2023-15:14:23] [V] Throughput: the observed throughput computed by dividing the number of queries by the Total Host Walltime. If this is significantly lower than the reciprocal of GPU Compute Time, the GPU may be under-utilized because of host-side overheads or data transfers. [05/12/2023-15:14:23] [V] Enqueue Time: the host latency to enqueue a query. If this is longer than GPU Compute Time, the GPU may be under-utilized. [05/12/2023-15:14:23] [V] H2D Latency: the latency for host-to-device data transfers for input tensors of a single query. [05/12/2023-15:14:23] [V] D2H Latency: the latency for device-to-host data transfers for output tensors of a single query. [05/12/2023-15:14:23] [V] Latency: the summation of H2D Latency, GPU Compute Time, and D2H Latency. This is the latency to infer a single query. [05/12/2023-15:14:23] [I] &&&& PASSED TensorRT.trtexec [TensorRT v8500] # trtexec --onnx=model.onnx --saveEngine=model.plan --minShapes=token_ids:1x128,attn_mask:1x128 --optShapes=token_ids:16x128,attn_mask:16x128 --maxShapes=token_ids:32x128,attn_mask:32x128 --verbose --workspace=14000