/var/opt/nydus/ops/mysql/ai/genai/__pycache__
Edit: /var/opt/nydus/ops/mysql/ai/genai/__pycache__/vector_store.cpython-312.pyc (19482B)
N jzH d Z ddlZddlmZmZmZmZmZmZ ddl Z
ddl m Z ddl
mZ ddlmZ ddlmZ ddlmZ dd lmZmZmZmZmZmZm Z m Z m Z m Z dd
l m!Z! d Z"d Z#e d
Z$e dZ%e dZ&e dZ'e dZ(dZ) G d de Z*y)zMySQL-backed vector store for embeddings and semantic document retrieval.
Provides a VectorStore implementation persisting documents, metadata, and
embeddings in MySQL, plus similarity search utilities.
N)AnyIterableListOptionalSequenceUnion)Document)
Embeddings) VectorStore) PrivateAttr) MyEmbeddings)
VAR_NAME_SPACEatomic_transactiondelete_sql_table execute_sqlextend_sql_tableformat_value_sqlget_random_nameis_table_empty
source_schema table_exists)MySQLConnectionAbstractz Hello world!external_sourcez
.embeddingz.contextz .context_mapz.retrieval_infoz.optionsinternal_ai_id_c ( e Zd ZU dZ e Zeed< e Ze ed< e Z
e ed< e Z e
e ed< e Zeed< e Zeed< d#d ed
e
e d df fd
Zd
ed ee fdZd$dZd#de
ee ded dfdZd$dZ d%dee de
ee de
ee ded ee f
dZ e d%dee d
e de
ee d ed e f
d Z d#dee dee d ee fdZ! d&de ded ed ee fd Z"d'd Z#d e$e%df d e$e&df d!e$e'df d dfd"Z( xZ)S )(
MyVectorStorea
MySQL-backed vector store for handling embeddings and semantic document retrieval.
Supports adding, deleting, and searching high-dimensional vector representations
of documents using efficient storage and HeatWave ML similarity search procedures.
Supports use as a context manager: when used in a `with` statement, all backing
tables/data are deleted automatically when the block exits (even on exception).
Attributes:
db_connection (MySQLConnectionAbstract): Active MySQL database connection.
embedder (Embeddings): Embeddings generator for computing vector representations.
schema_name (str): SQL schema for table storage.
table_name (Optional[str]): Name of the active table backing the store
(or None until created).
embedding_dimension (int): Size of embedding vectors stored.
next_id (int): Internal counter for unique document ID generation.
_db_connection _embedder _schema_name _table_name_embedding_dimension_next_idN
db_connectionembedderreturnc t | d| _ t | | _ |xs t | | _ || _ d| _ t | j j t | _ y)a
Initialize a MyVectorStore with a database connection and embedding generator.
Args:
db_connection: MySQL database connection for all vector operations.
embedder: Embeddings generator used for creating and querying embeddings.
Raises:
ValueError: If the schema name is not valid
DatabaseError:
If a database connection issue occurs.
If an operational error occurs during execution.
r N)
super__init__r" r r r
r r r len embed_queryBASIC_EMBEDDING_QUERYr! )selfr# r$ __class__s L/opt/nydus/tmp/pip-target-a90h98xg/lib/python/mysql/ai/genai/vector_store.pyr( zMyVectorStore.__init__d sg $
)- 8!@\-%@ +*. %( NN & &'< =%
! num_idsc t | j | j |z D cg c] }d| }}| xj |z
c_ |S c c}w )z
Generate a batch of unique internal document IDs for vector storage.
Args:
num_ids: Number of IDs to create.
Returns:
List of sequentially numbered internal string IDs.
r )ranger" )r, r0 iidss r. _get_idszMyVectorStore._get_ids sW ,1
PW@W+X
+XaoaS !+X
s A c j at j 5 t fd }d j d| d}t | j f | _ ddd yy# 1 sw Y yxY w)a
Create a backing SQL table for storing vectors if not already created.
Returns:
None
Raises:
DatabaseError:
If a database connection issue occurs.
If an operational error occurs during execution.
Notes:
The table name is randomized to avoid collisions.
Schema includes content, metadata, and embedding vector.
Nc 4 t j | S N)r r )
table_namecursorr, s r.
z2MyVectorStore._make_vector_store.. s < 1 1:, (r/ z
CREATE TABLE .a (
`id` VARCHAR(128) NOT NULL,
`content` TEXT,
`metadata` JSON DEFAULT NULL,
`embed` vector(%s),
PRIMARY KEY (`id`)
) ENGINE=InnoDB;
params)r r r r r r r! )r, r9 create_table_stmtr: s ` @r. _make_vector_storez MyVectorStore._make_vector_store s ##D$7$78F ,
) " / / 0* >%! -t7P7P6R $. ) 98 $88s AA11A:r4 _c
t | j 5 }|r1|D ], }t |d| j d| j d|f . t || j | j r| j
ddd y# 1 sw Y yxY w)a#
Delete documents by ID. Optionally deletes the vector table if empty after deletions.
Args:
ids: Optional sequence of document IDs to delete. If None, no action is taken.
Returns:
None
Raises:
DatabaseError:
If a database connection issue occurs.
If an operational error occurs during execution.
Notes:
If the backing table is empty after deletions, the table is dropped and
table_name is set to None.
z DELETE FROM r< WHERE id = %sr= N)r r r r r r
delete_all)r, r4 rA r: _ids r. deletezMyVectorStore.delete s & 3 3
4 C &t'8'8&94;K;K:LN[ #v fd&7&79I9IJ! 5
4
4s A%BB
c | j Gt | j 5 }t || j | j d| _ ddd yy# 1 sw Y yxY w)zc
Delete and drop the entire vector store table.
Returns:
None
N)r r r r r )r, r: s r. rD zMyVectorStore.delete_all sT '#D$7$78F ):):DA # # %I ;--I/C/89y3##y9..009
\\^4 9
7 2;<)3#,,)<:
3 3
4 !! 079O/P
5
% : =
4
s D7D<'EE querykkwargsc | j g S | j j | }t | j 5 }t |dt dt | g |j dd }|j dd |j d d
|j d d d
}t | \ }} dt d| j d| j d| d| dt dt dt d}
t ||
|g| t |dt g } t j |j! d } | D ]~ }
t |d| j d| j d|
d f |j! \ }}}||d}|t j | |d< t# di |}| j% | | cddd S # 1 sw Y yxY w) a
Search for and return the most similar documents in the store to the given query.
Args:
query: String query to embed and use for similarity search.
k: Number of top documents to return.
kwargs: options to pass to ML_SIMILARITY_SEARCH. Currently supports
distance_metric, max_distance, percentage_distance, and segment_overlap
Returns:
List of Document objects, ordered from most to least similar.
Raises:
DatabaseError:
If provided kwargs are invalid or unsupported.
If a database connection issue occurs.
If an operational error occurs during execution.
Implementation Notes:
- Calls ML similarity search within MySQL using stored procedures.
- Retrieves IDs, content, and metadata for search matches.
- Parsing and retrieval for context results are handled via intermediate JSONs.
NzSET @z = string_to_vector(%s)r= distance_metricCOSINE max_distanceg333333?percentage_distanceg 4@segment_overlapr )rk rl rm z=
CALL sys.ML_SIMILARITY_SEARCH(
@z3,
JSON_ARRAY(
'r< z'
),
JSON_OBJECT(
"segment", "content",
"segment_embedding", "embed",
"document_name", "id"
),
zR,
%s,
NULL,
NULL,
z,
@z
)
zSELECT @z"SELECT id, content, metadata FROM rC
document_name)rZ rK rL )r r r* r r r
VAR_EMBEDDINGstrgetr r VAR_CONTEXTVAR_CONTEXT_MAPVAR_RETRIEVAL_INFOjsonloadsfetchoner append)r, re rf rg embeddingr: ri retrieval_options retrieval_options_placeholderretrieval_options_paramssimilarity_search_queryresults context_mapscontextdoc_idr[ rL doc_argsrb s r. similarity_searchz MyVectorStore.similarity_searchg s- : #INN..u5
3 3
4
&=>I'
%jj):H EO &
>3 ?'-zz2G'N#)::.?#C! GW!G C )+C+ !''($*:*:); < // 0 !" #$% &
%' #,
''C*BC
(?*; < =G ::foo&7&:;L' $ 1 1 2!D4D4D3E F(( $O 46 -3OO,=) !$+ '+/::h+?HZ( **s#) (, Q 5
4
4s FGGc | S )a
Enter the runtime context related to this vector store instance.
Returns:
The current MyVectorStore object, allowing use within a `with` statement block.
Usage Notes:
- Intended for use in a `with` statement to ensure automatic
cleanup of resources.
- No special initialization occurs during context entry, but enables
proper context-managed lifecycle.
Example:
with MyVectorStore(db_connection, embedder) as vectorstore:
vectorstore.add_texts([...])
# Vector store is active within this block.
# All storage and resources are now cleaned up.
ro )r, s r. __enter__zMyVectorStore.__enter__ s & r/ exc_typeexc_valexc_tbc $ | j y)a
Exit the runtime context for the vector store, ensuring all storage
resources are cleaned up.
Args:
exc_type: The exception type, if any exception occurred in the context block.
exc_val: The exception value, if any exception occurred in the context block.
exc_tb: The traceback object, if any exception occurred in the context block.
Returns:
None: Indicates that exceptions are never suppressed; they will propagate as normal.
Implementation Notes:
- Automatically deletes all vector store data and backing tables via `delete_all()`
upon exiting the context.
- This cleanup occurs whether the block exits normally or due to an exception.
- Does not suppress exceptions; errors in the context block will continue to propagate.
- Use when the vector store lifecycle is intended to be temporary or scoped.
Example:
with MyVectorStore(db_connection, embedder) as vectorstore:
vectorstore.add_texts([...])
# Vector store is active within this block.
# All storage and resources are now cleaned up.
N)rD )r, r r r s r. __exit__zMyVectorStore.__exit__ s >
r/ r8 )r% N)NN) )r% r )*__name__
__module__ __qualname____doc__r r r __annotations__r r
r rq r r r! intr" r( rM r5 r@ r r rF rD r dictr rS classmethodr rX r rO r r r type
BaseExceptionobjectr
__classcell__)r- s @r. r r I s0 & /:mN+; 'MIz)#
L#%!, K# . +
#- MHc!
*.
.
:&
@ S %.N "(8C=1 "s "t ">
( +/#' 6} 6 DJ' 6 d3i
6
6
c
6D
+/15 #}# # DJ' #
/ #
# #L ;?7 h 7.23i7
c7x jj j j
h jX* d
# }d*+ fdl#
r/ r )+r rv typingr r r r r r pandasr_ langchain_core.documentsr langchain_core.embeddingsr
langchain_core.vectorstoresr pydanticr mysql.ai.genai.embeddingr
mysql.ai.utilsr r r r r r r r r r mysql.connector.abstractsr r+ EMBEDDING_SOURCErp rs rt ru VAR_OPTIONSID_SPACEr ro r/ r. r s : A A - 0 3 1 >& $ !"*-
) #$L1&'7 ) ~K ~r/