Skip to content

vllm.transformers_utils.configs.colpali

ColPali configuration that extends PaliGemmaConfig with embedding projection fields. This allows ColPali models to be loaded without trust_remote_code by mapping their custom model_type (colpali) to a standard config class that vLLM understands.

Supported model_types: - colpali (vidore/colpali-v1.3-hf)

ColPaliConfig

Bases: PaliGemmaConfig

Configuration class for ColPali models.

Extends PaliGemmaConfig with additional fields used by ColPali variants for the embedding projection layer.

Source code in vllm/transformers_utils/configs/colpali.py 
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
class ColPaliConfig(PaliGemmaConfig):
    """Configuration class for ColPali models.

    Extends PaliGemmaConfig with additional fields used by ColPali variants
    for the embedding projection layer.
    """

    model_type = "colpali"

    def __init__(
        self,
        embedding_dim: int | None = None,
        embed_dim: int | None = None,
        dim: int | None = None,
        projection_dim: int | None = None,
        colbert_dim: int | None = None,
        pooling: str | None = None,
        vlm_config: dict | None = None,
        **kwargs,
    ):
        # Store embedding projection config fields
        self.embedding_dim = embedding_dim
        self.embed_dim = embed_dim
        self.dim = dim
        self.projection_dim = projection_dim
        self.colbert_dim = colbert_dim
        self.pooling = pooling

        # The HF checkpoint nests PaliGemma config inside "vlm_config".
        # Flatten it so PaliGemmaConfig receives vision_config, text_config,
        # image_token_index, etc. directly.
        if vlm_config is not None:
            if isinstance(vlm_config, dict):
                kwargs.update(vlm_config)
            else:
                kwargs.update(vlm_config.to_dict())

        super().__init__(**kwargs)