Skip to content

Confidence is not returned in the decoding example? #783

@liuzhuang1024

Description

@liuzhuang1024

Branch/Tag/Commit

main

Docker Image Version

nvcr.io/nvidia/pytorch:22.08-py3

GPU name

A100

CUDA Driver

470.161.03

Reproduced Steps

In src/fastertransformer/models/decoding/Decoding.h:134

    Decoding(Decoding<T> const& Decoding);

    ~Decoding();

    void forward(std::vector<Tensor>*       output_tensors,
                 const std::vector<Tensor>* input_tensors,
                 const DecodingWeight<T>*   Decoding_weights);

No corresponding confidence information is sent from this place?

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions