Skip to content

InformationWeightTransform hates empty columns #101

@jc-healy

Description

@jc-healy

We seem to crash kernels when we pass a sparse matrix with an empty column to InformationWeight transformers fit transform.

It comes up when we've got a fixed token dictionary but our training data is missing some of our vocabulary:

vect = NgramVectorizer(token_dictionary=token_dictionary).fit_transform(eventid_list)
vect1 = InformationWeightTransformer().fit_transform(vect)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions