Esc
Ask AIAnswers may be inaccurate; check the linked pages.Esc
Ask anything about these docs, like how to get started or what a function does.

SwiGLU

feed_forward.SwiGLU

https://arxiv.org/abs/2002.05202 NOTE: x should be 2x the size you want

forward(self, x: torch.Tensor) -> torch.Tensor

(undocumented)

Parameter Type Default Description
x torch.Tensor - (undocumented)