num_keep defines how many tokens to keep in the context when truncating inputs. if left to its default value of -1, the server will calculate num_keep to be the left of the system instructions |
||
|---|---|---|
| .. | ||
| images.go | ||
| modelpath.go | ||
| routes.go | ||
num_keep defines how many tokens to keep in the context when truncating inputs. if left to its default value of -1, the server will calculate num_keep to be the left of the system instructions |
||
|---|---|---|
| .. | ||
| images.go | ||
| modelpath.go | ||
| routes.go | ||