Extensions
pg_tokenizer

pg_tokenizer

pg_tokenizer : Tokenizers for full-text search

Overview

ID Extension Package Version Category License Language
2160
pg_tokenizer
pg_tokenizer
 0.1.1
FTS
Apache-2.0
Rust
Attribute Has Binary Has Library Need Load Has DDL Relocatable Trusted
--s-dt-
No
Yes
No
Yes
no
yes
Relationships
Schemas tokenizer_catalog
See Also
pg_search
pgroonga
pg_bigm
zhparser
pgroonga_database
pg_bestmatch
vchord_bm25
pg_trgm

PG18 fix by Vonng

Packages

Type Repo Version PG Major Compatibility Package Pattern Dependencies
EXT
PIGSTY
 0.1.1
18
17
16
15
14
 pg_tokenizer -
RPM
PIGSTY
 0.1.1
18
17
16
15
14
 pg_tokenizer_$v -
DEB
PIGSTY
 0.1.1
18
17
16
15
14
 postgresql-$v-pg-tokenizer -
Linux / PG PG18 PG17 PG16 PG15 PG14
el8.x86_64
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
el8.aarch64
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
el9.x86_64
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
el9.aarch64
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
el10.x86_64
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
el10.aarch64
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
d12.x86_64
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
d12.aarch64
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
d13.x86_64
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
d13.aarch64
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
u22.x86_64
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
u22.aarch64
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
u24.x86_64
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
u24.aarch64
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
PIGSTY 0.1.1
Package Version OS ORG SIZE File URL
pg_tokenizer_18 0.1.1 el8.x86_64 pigsty 11.7 MiB pg_tokenizer_18-0.1.1-1PIGSTY.el8.x86_64.rpm
pg_tokenizer_18 0.1.1 el8.aarch64 pigsty 11.5 MiB pg_tokenizer_18-0.1.1-1PIGSTY.el8.aarch64.rpm
pg_tokenizer_18 0.1.1 el9.x86_64 pigsty 11.0 MiB pg_tokenizer_18-0.1.1-1PIGSTY.el9.x86_64.rpm
pg_tokenizer_18 0.1.1 el9.aarch64 pigsty 10.9 MiB pg_tokenizer_18-0.1.1-1PIGSTY.el9.aarch64.rpm
pg_tokenizer_18 0.1.1 el10.x86_64 pigsty 10.9 MiB pg_tokenizer_18-0.1.1-1PIGSTY.el10.x86_64.rpm
pg_tokenizer_18 0.1.1 el10.aarch64 pigsty 11.0 MiB pg_tokenizer_18-0.1.1-1PIGSTY.el10.aarch64.rpm
postgresql-18-pg-tokenizer 0.1.1 d12.x86_64 pigsty 9.9 MiB postgresql-18-pg-tokenizer_0.1.1-1PIGSTY~bookworm_amd64.deb
postgresql-18-pg-tokenizer 0.1.1 d12.aarch64 pigsty 9.7 MiB postgresql-18-pg-tokenizer_0.1.1-1PIGSTY~bookworm_arm64.deb
postgresql-18-pg-tokenizer 0.1.1 d13.x86_64 pigsty 9.9 MiB postgresql-18-pg-tokenizer_0.1.1-1PIGSTY~trixie_amd64.deb
postgresql-18-pg-tokenizer 0.1.1 d13.aarch64 pigsty 9.7 MiB postgresql-18-pg-tokenizer_0.1.1-1PIGSTY~trixie_arm64.deb
postgresql-18-pg-tokenizer 0.1.1 u22.x86_64 pigsty 10.9 MiB postgresql-18-pg-tokenizer_0.1.1-1PIGSTY~jammy_amd64.deb
postgresql-18-pg-tokenizer 0.1.1 u22.aarch64 pigsty 10.7 MiB postgresql-18-pg-tokenizer_0.1.1-1PIGSTY~jammy_arm64.deb
postgresql-18-pg-tokenizer 0.1.1 u24.x86_64 pigsty 10.8 MiB postgresql-18-pg-tokenizer_0.1.1-1PIGSTY~noble_amd64.deb
postgresql-18-pg-tokenizer 0.1.1 u24.aarch64 pigsty 10.6 MiB postgresql-18-pg-tokenizer_0.1.1-1PIGSTY~noble_arm64.deb
Package Version OS ORG SIZE File URL
pg_tokenizer_17 0.1.1 el8.x86_64 pigsty 11.7 MiB pg_tokenizer_17-0.1.1-1PIGSTY.el8.x86_64.rpm
pg_tokenizer_17 0.1.1 el8.aarch64 pigsty 11.5 MiB pg_tokenizer_17-0.1.1-1PIGSTY.el8.aarch64.rpm
pg_tokenizer_17 0.1.1 el9.x86_64 pigsty 11.0 MiB pg_tokenizer_17-0.1.1-1PIGSTY.el9.x86_64.rpm
pg_tokenizer_17 0.1.1 el9.aarch64 pigsty 10.9 MiB pg_tokenizer_17-0.1.1-1PIGSTY.el9.aarch64.rpm
pg_tokenizer_17 0.1.1 el10.x86_64 pigsty 10.9 MiB pg_tokenizer_17-0.1.1-1PIGSTY.el10.x86_64.rpm
pg_tokenizer_17 0.1.1 el10.aarch64 pigsty 11.0 MiB pg_tokenizer_17-0.1.1-1PIGSTY.el10.aarch64.rpm
postgresql-17-pg-tokenizer 0.1.1 d12.x86_64 pigsty 9.9 MiB postgresql-17-pg-tokenizer_0.1.1-1PIGSTY~bookworm_amd64.deb
postgresql-17-pg-tokenizer 0.1.1 d12.aarch64 pigsty 9.7 MiB postgresql-17-pg-tokenizer_0.1.1-1PIGSTY~bookworm_arm64.deb
postgresql-17-pg-tokenizer 0.1.1 d13.x86_64 pigsty 9.9 MiB postgresql-17-pg-tokenizer_0.1.1-1PIGSTY~trixie_amd64.deb
postgresql-17-pg-tokenizer 0.1.1 d13.aarch64 pigsty 9.7 MiB postgresql-17-pg-tokenizer_0.1.1-1PIGSTY~trixie_arm64.deb
postgresql-17-pg-tokenizer 0.1.1 u22.x86_64 pigsty 10.9 MiB postgresql-17-pg-tokenizer_0.1.1-1PIGSTY~jammy_amd64.deb
postgresql-17-pg-tokenizer 0.1.1 u22.aarch64 pigsty 10.7 MiB postgresql-17-pg-tokenizer_0.1.1-1PIGSTY~jammy_arm64.deb
postgresql-17-pg-tokenizer 0.1.1 u24.x86_64 pigsty 10.8 MiB postgresql-17-pg-tokenizer_0.1.1-1PIGSTY~noble_amd64.deb
postgresql-17-pg-tokenizer 0.1.1 u24.aarch64 pigsty 10.7 MiB postgresql-17-pg-tokenizer_0.1.1-1PIGSTY~noble_arm64.deb
Package Version OS ORG SIZE File URL
pg_tokenizer_16 0.1.1 el8.x86_64 pigsty 11.7 MiB pg_tokenizer_16-0.1.1-1PIGSTY.el8.x86_64.rpm
pg_tokenizer_16 0.1.1 el8.aarch64 pigsty 11.5 MiB pg_tokenizer_16-0.1.1-1PIGSTY.el8.aarch64.rpm
pg_tokenizer_16 0.1.1 el9.x86_64 pigsty 11.0 MiB pg_tokenizer_16-0.1.1-1PIGSTY.el9.x86_64.rpm
pg_tokenizer_16 0.1.1 el9.aarch64 pigsty 10.9 MiB pg_tokenizer_16-0.1.1-1PIGSTY.el9.aarch64.rpm
pg_tokenizer_16 0.1.1 el10.x86_64 pigsty 10.9 MiB pg_tokenizer_16-0.1.1-1PIGSTY.el10.x86_64.rpm
pg_tokenizer_16 0.1.1 el10.aarch64 pigsty 11.0 MiB pg_tokenizer_16-0.1.1-1PIGSTY.el10.aarch64.rpm
postgresql-16-pg-tokenizer 0.1.1 d12.x86_64 pigsty 9.9 MiB postgresql-16-pg-tokenizer_0.1.1-1PIGSTY~bookworm_amd64.deb
postgresql-16-pg-tokenizer 0.1.1 d12.aarch64 pigsty 9.7 MiB postgresql-16-pg-tokenizer_0.1.1-1PIGSTY~bookworm_arm64.deb
postgresql-16-pg-tokenizer 0.1.1 d13.x86_64 pigsty 9.9 MiB postgresql-16-pg-tokenizer_0.1.1-1PIGSTY~trixie_amd64.deb
postgresql-16-pg-tokenizer 0.1.1 d13.aarch64 pigsty 9.7 MiB postgresql-16-pg-tokenizer_0.1.1-1PIGSTY~trixie_arm64.deb
postgresql-16-pg-tokenizer 0.1.1 u22.x86_64 pigsty 10.9 MiB postgresql-16-pg-tokenizer_0.1.1-1PIGSTY~jammy_amd64.deb
postgresql-16-pg-tokenizer 0.1.1 u22.aarch64 pigsty 10.7 MiB postgresql-16-pg-tokenizer_0.1.1-1PIGSTY~jammy_arm64.deb
postgresql-16-pg-tokenizer 0.1.1 u24.x86_64 pigsty 10.8 MiB postgresql-16-pg-tokenizer_0.1.1-1PIGSTY~noble_amd64.deb
postgresql-16-pg-tokenizer 0.1.1 u24.aarch64 pigsty 10.7 MiB postgresql-16-pg-tokenizer_0.1.1-1PIGSTY~noble_arm64.deb
Package Version OS ORG SIZE File URL
pg_tokenizer_15 0.1.1 el8.x86_64 pigsty 11.7 MiB pg_tokenizer_15-0.1.1-1PIGSTY.el8.x86_64.rpm
pg_tokenizer_15 0.1.1 el8.aarch64 pigsty 11.5 MiB pg_tokenizer_15-0.1.1-1PIGSTY.el8.aarch64.rpm
pg_tokenizer_15 0.1.1 el9.x86_64 pigsty 11.0 MiB pg_tokenizer_15-0.1.1-1PIGSTY.el9.x86_64.rpm
pg_tokenizer_15 0.1.1 el9.aarch64 pigsty 10.9 MiB pg_tokenizer_15-0.1.1-1PIGSTY.el9.aarch64.rpm
pg_tokenizer_15 0.1.1 el10.x86_64 pigsty 10.9 MiB pg_tokenizer_15-0.1.1-1PIGSTY.el10.x86_64.rpm
pg_tokenizer_15 0.1.1 el10.aarch64 pigsty 11.0 MiB pg_tokenizer_15-0.1.1-1PIGSTY.el10.aarch64.rpm
postgresql-15-pg-tokenizer 0.1.1 d12.x86_64 pigsty 9.9 MiB postgresql-15-pg-tokenizer_0.1.1-1PIGSTY~bookworm_amd64.deb
postgresql-15-pg-tokenizer 0.1.1 d12.aarch64 pigsty 9.7 MiB postgresql-15-pg-tokenizer_0.1.1-1PIGSTY~bookworm_arm64.deb
postgresql-15-pg-tokenizer 0.1.1 d13.x86_64 pigsty 9.9 MiB postgresql-15-pg-tokenizer_0.1.1-1PIGSTY~trixie_amd64.deb
postgresql-15-pg-tokenizer 0.1.1 d13.aarch64 pigsty 9.7 MiB postgresql-15-pg-tokenizer_0.1.1-1PIGSTY~trixie_arm64.deb
postgresql-15-pg-tokenizer 0.1.1 u22.x86_64 pigsty 10.9 MiB postgresql-15-pg-tokenizer_0.1.1-1PIGSTY~jammy_amd64.deb
postgresql-15-pg-tokenizer 0.1.1 u22.aarch64 pigsty 10.7 MiB postgresql-15-pg-tokenizer_0.1.1-1PIGSTY~jammy_arm64.deb
postgresql-15-pg-tokenizer 0.1.1 u24.x86_64 pigsty 10.8 MiB postgresql-15-pg-tokenizer_0.1.1-1PIGSTY~noble_amd64.deb
postgresql-15-pg-tokenizer 0.1.1 u24.aarch64 pigsty 10.7 MiB postgresql-15-pg-tokenizer_0.1.1-1PIGSTY~noble_arm64.deb
Package Version OS ORG SIZE File URL
pg_tokenizer_14 0.1.1 el8.x86_64 pigsty 11.7 MiB pg_tokenizer_14-0.1.1-1PIGSTY.el8.x86_64.rpm
pg_tokenizer_14 0.1.1 el8.aarch64 pigsty 11.5 MiB pg_tokenizer_14-0.1.1-1PIGSTY.el8.aarch64.rpm
pg_tokenizer_14 0.1.1 el9.x86_64 pigsty 11.0 MiB pg_tokenizer_14-0.1.1-1PIGSTY.el9.x86_64.rpm
pg_tokenizer_14 0.1.1 el9.aarch64 pigsty 10.9 MiB pg_tokenizer_14-0.1.1-1PIGSTY.el9.aarch64.rpm
pg_tokenizer_14 0.1.1 el10.x86_64 pigsty 10.9 MiB pg_tokenizer_14-0.1.1-1PIGSTY.el10.x86_64.rpm
pg_tokenizer_14 0.1.1 el10.aarch64 pigsty 11.0 MiB pg_tokenizer_14-0.1.1-1PIGSTY.el10.aarch64.rpm
postgresql-14-pg-tokenizer 0.1.1 d12.x86_64 pigsty 9.9 MiB postgresql-14-pg-tokenizer_0.1.1-1PIGSTY~bookworm_amd64.deb
postgresql-14-pg-tokenizer 0.1.1 d12.aarch64 pigsty 9.7 MiB postgresql-14-pg-tokenizer_0.1.1-1PIGSTY~bookworm_arm64.deb
postgresql-14-pg-tokenizer 0.1.1 d13.x86_64 pigsty 9.9 MiB postgresql-14-pg-tokenizer_0.1.1-1PIGSTY~trixie_amd64.deb
postgresql-14-pg-tokenizer 0.1.1 d13.aarch64 pigsty 9.7 MiB postgresql-14-pg-tokenizer_0.1.1-1PIGSTY~trixie_arm64.deb
postgresql-14-pg-tokenizer 0.1.1 u22.x86_64 pigsty 10.9 MiB postgresql-14-pg-tokenizer_0.1.1-1PIGSTY~jammy_amd64.deb
postgresql-14-pg-tokenizer 0.1.1 u22.aarch64 pigsty 10.7 MiB postgresql-14-pg-tokenizer_0.1.1-1PIGSTY~jammy_arm64.deb
postgresql-14-pg-tokenizer 0.1.1 u24.x86_64 pigsty 10.8 MiB postgresql-14-pg-tokenizer_0.1.1-1PIGSTY~noble_amd64.deb
postgresql-14-pg-tokenizer 0.1.1 u24.aarch64 pigsty 10.7 MiB postgresql-14-pg-tokenizer_0.1.1-1PIGSTY~noble_arm64.deb

Source

Repository
github.com/tensorchord/pg_tokenizer.rs
 Source Tarball
pg_tokenizer.rs-0.1.1.tar.gz
 
pig build pkg pg_tokenizer;		# build rpm/deb

Install

Make sure PGDG and PIGSTY repo available:

pig repo add pgsql -u   # add both repo and update cache

Install this extension with pig:

pig install pg_tokenizer;		# install via package name, for the active PG version

pig install pg_tokenizer -v 18;   # install for PG 18
pig install pg_tokenizer -v 17;   # install for PG 17
pig install pg_tokenizer -v 16;   # install for PG 16
pig install pg_tokenizer -v 15;   # install for PG 15
pig install pg_tokenizer -v 14;   # install for PG 14

Create this extension with:

CREATE EXTENSION pg_tokenizer;
Last updated on
vchord_bm25biscuit