hanzceo commited on
Commit
6b2fbf2
·
verified ·
1 Parent(s): 6da271e

Add new SentenceTransformer model

Browse files
README.md ADDED
@@ -0,0 +1,598 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: wtfpl
5
+ tags:
6
+ - sentence-transformers
7
+ - sentence-similarity
8
+ - feature-extraction
9
+ - generated_from_trainer
10
+ - dataset_size:4373977
11
+ - loss:MatryoshkaLoss
12
+ - loss:MultipleNegativesRankingLoss
13
+ widget:
14
+ - source_sentence: crack
15
+ sentences:
16
+ - master
17
+ - Academic journals often publish both print editions and digital versions simultaneously.
18
+ - ceramic ware
19
+ - source_sentence: An area where a specific ethnic group maintains distinctive social
20
+ and cultural institutions within the larger city.
21
+ sentences:
22
+ - foot
23
+ - Policy makers study ethnic enclaves to understand immigrant entrepreneurship.
24
+ - economic entity
25
+ - source_sentence: We lease various facilities, office equipment, plant equipment
26
+ and railcars under operating leases.
27
+ sentences:
28
+ - Under the ESPP, qualified employees are permitted to invest as much as ten percent
29
+ of their base pay to acquire company common stock at a discount of 15 percent
30
+ off the closing market price on the day of purchase.
31
+ - On March 1, 2006, the total number of full-time staff members across ACGL and
32
+ its subsidiary companies was roughly 980.
33
+ - Our company utilizes operating leases for the rental of various types of equipment,
34
+ including railcars, plant machinery, office gear, and various facilities.
35
+ - source_sentence: How will the ban of Rs 500 and Rs 1000 notes affect Indian economy?
36
+ sentences:
37
+ - Where can I get professional photo booths service in Sydney?
38
+ - What are your views on India banning 500 and 1000 notes? In what way it will affect
39
+ Indian economy?
40
+ - What are some of your favorite sayings?
41
+ - source_sentence: How do you write a song?
42
+ sentences:
43
+ - How did Portugal become an independent country from Spain?
44
+ - Who created the "GOD"?
45
+ - How do I write a song?
46
+ pipeline_tag: sentence-similarity
47
+ library_name: sentence-transformers
48
+ ---
49
+
50
+ # PurpleStatic: Static Embeddings
51
+
52
+ This is a [sentence-transformers](https://www.SBERT.net) model trained on the BatuhanECB/FinModernBERT-pairs-sec-synthetic-v1, Heliosoph/Quora-Question-Pairs, owenkaplinsky/wildchat-paraphrases, mjbommar/ogbert-v1-contrastive and mjbommar/opengloss-v1.3-contrastive-examples datasets. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for retrieval.
53
+
54
+ ## Model Details
55
+
56
+ ### Model Description
57
+ - **Model Type:** Sentence Transformer
58
+ <!-- - **Base model:** [Unknown](https://huggingface.co/unknown) -->
59
+ - **Maximum Sequence Length:** inf tokens
60
+ - **Output Dimensionality:** 1024 dimensions
61
+ - **Similarity Function:** Cosine Similarity
62
+ - **Supported Modality:** Text
63
+ - **Training Datasets:**
64
+ - BatuhanECB/FinModernBERT-pairs-sec-synthetic-v1
65
+ - Heliosoph/Quora-Question-Pairs
66
+ - owenkaplinsky/wildchat-paraphrases
67
+ - mjbommar/ogbert-v1-contrastive
68
+ - mjbommar/opengloss-v1.3-contrastive-examples
69
+ - **Language:** en
70
+ - **License:** wtfpl
71
+
72
+ ### Model Sources
73
+
74
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
75
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)
76
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
77
+
78
+ ### Full Model Architecture
79
+
80
+ ```
81
+ SentenceTransformer(
82
+ (0): StaticEmbedding({})
83
+ )
84
+ ```
85
+
86
+ ## Usage
87
+
88
+ ### Direct Usage (Sentence Transformers)
89
+
90
+ First install the Sentence Transformers library:
91
+
92
+ ```bash
93
+ pip install -U sentence-transformers
94
+ ```
95
+ Then you can load this model and run inference.
96
+ ```python
97
+ from sentence_transformers import SentenceTransformer
98
+
99
+ # Download from the 🤗 Hub
100
+ model = SentenceTransformer("LocalWisdom/PurpleStatic")
101
+ # Run inference
102
+ sentences = [
103
+ 'How do you write a song?',
104
+ 'How do I write a song?',
105
+ 'How did Portugal become an independent country from Spain?',
106
+ ]
107
+ embeddings = model.encode(sentences)
108
+ print(embeddings.shape)
109
+ # [3, 1024]
110
+
111
+ # Get the similarity scores for the embeddings
112
+ similarities = model.similarity(embeddings, embeddings)
113
+ print(similarities)
114
+ # tensor([[1.0000, 0.8566, 0.1409],
115
+ # [0.8566, 1.0000, 0.1259],
116
+ # [0.1409, 0.1259, 1.0000]])
117
+ ```
118
+ <!--
119
+ ### Direct Usage (Transformers)
120
+
121
+ <details><summary>Click to see the direct usage in Transformers</summary>
122
+
123
+ </details>
124
+ -->
125
+
126
+ <!--
127
+ ### Downstream Usage (Sentence Transformers)
128
+
129
+ You can finetune this model on your own dataset.
130
+
131
+ <details><summary>Click to expand</summary>
132
+
133
+ </details>
134
+ -->
135
+
136
+ <!--
137
+ ### Out-of-Scope Use
138
+
139
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
140
+ -->
141
+
142
+ <!--
143
+ ## Bias, Risks and Limitations
144
+
145
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
146
+ -->
147
+
148
+ <!--
149
+ ### Recommendations
150
+
151
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
152
+ -->
153
+
154
+ ## Training Details
155
+
156
+ ### Training Datasets
157
+ <details><summary>BatuhanECB/FinModernBERT-pairs-sec-synthetic-v1</summary>
158
+
159
+ #### BatuhanECB/FinModernBERT-pairs-sec-synthetic-v1
160
+
161
+ * Dataset: BatuhanECB/FinModernBERT-pairs-sec-synthetic-v1
162
+ * Size: 5,694 training samples
163
+ * Columns: <code>anchor</code> and <code>positive</code>
164
+ * Approximate statistics based on the first 100 samples:
165
+ | | anchor | positive |
166
+ |:---------|:-------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------|
167
+ | type | string | string |
168
+ | modality | text | text |
169
+ | details | <ul><li>min: 47 characters</li><li>mean: 135.34 characters</li><li>max: 352 characters</li></ul> | <ul><li>min: 70 characters</li><li>mean: 146.17 characters</li><li>max: 322 characters</li></ul> |
170
+ * Samples:
171
+ | anchor | positive |
172
+ |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
173
+ | <code>Of our total net revenue of $10.8 billion in the fiscal year ended October 31, 2000, we generated 44.2% in the United States and 55.8% internationally.</code> | <code>In the fiscal year concluding October 31, 2000, the company produced 44.2% of its $10.8 billion total net revenue within the United States and 55.8% from international markets.</code> |
174
+ | <code>Our sales strategy is to sell to and service our largest accounts (hospital and corporate business) directly while employing third-party distributors and manufacturer's representatives for smaller or more geographically dispersed countries.</code> | <code>We utilize a sales approach where we directly manage and sell to our major hospital and corporate clients, while relying on external distributors and manufacturer agents to reach smaller markets or those in remote locations.</code> |
175
+ | <code>In August 2000, we announced a restructuring of our healthcare solutions business.</code> | <code>The company declared a reorganization of its healthcare solutions division in August 2000.</code> |
176
+ * Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
177
+ ```json
178
+ {
179
+ "loss": "MultipleNegativesRankingLoss",
180
+ "matryoshka_dims": [
181
+ 1024,
182
+ 768,
183
+ 512,
184
+ 256,
185
+ 128,
186
+ 64,
187
+ 32
188
+ ],
189
+ "matryoshka_weights": [
190
+ 1,
191
+ 1,
192
+ 1,
193
+ 1,
194
+ 1,
195
+ 1,
196
+ 1
197
+ ],
198
+ "n_dims_per_step": -1
199
+ }
200
+ ```
201
+ </details>
202
+ <details><summary>Heliosoph/Quora-Question-Pairs</summary>
203
+
204
+ #### Heliosoph/Quora-Question-Pairs
205
+
206
+ * Dataset: Heliosoph/Quora-Question-Pairs
207
+ * Size: 149,263 training samples
208
+ * Columns: <code>anchor</code> and <code>positive</code>
209
+ * Approximate statistics based on the first 100 samples:
210
+ | | anchor | positive |
211
+ |:---------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------|
212
+ | type | string | string |
213
+ | modality | text | text |
214
+ | details | <ul><li>min: 16 characters</li><li>mean: 54.75 characters</li><li>max: 139 characters</li></ul> | <ul><li>min: 21 characters</li><li>mean: 54.34 characters</li><li>max: 127 characters</li></ul> |
215
+ * Samples:
216
+ | anchor | positive |
217
+ |:----------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------|
218
+ | <code>Astrology: I am a Capricorn Sun Cap moon and cap rising...what does that say about me?</code> | <code>I'm a triple Capricorn (Sun, Moon and ascendant in Capricorn) What does this say about me?</code> |
219
+ | <code>How can I be a good geologist?</code> | <code>What should I do to be a great geologist?</code> |
220
+ | <code>How do I read and find my YouTube comments?</code> | <code>How can I see all my Youtube comments?</code> |
221
+ * Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
222
+ ```json
223
+ {
224
+ "loss": "MultipleNegativesRankingLoss",
225
+ "matryoshka_dims": [
226
+ 1024,
227
+ 768,
228
+ 512,
229
+ 256,
230
+ 128,
231
+ 64,
232
+ 32
233
+ ],
234
+ "matryoshka_weights": [
235
+ 1,
236
+ 1,
237
+ 1,
238
+ 1,
239
+ 1,
240
+ 1,
241
+ 1
242
+ ],
243
+ "n_dims_per_step": -1
244
+ }
245
+ ```
246
+ </details>
247
+ <details><summary>owenkaplinsky/wildchat-paraphrases</summary>
248
+
249
+ #### owenkaplinsky/wildchat-paraphrases
250
+
251
+ * Dataset: owenkaplinsky/wildchat-paraphrases
252
+ * Size: 1,383,750 training samples
253
+ * Columns: <code>anchor</code> and <code>positive</code>
254
+ * Approximate statistics based on the first 100 samples:
255
+ | | anchor | positive |
256
+ |:---------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------|
257
+ | type | string | string |
258
+ | modality | text | text |
259
+ | details | <ul><li>min: 20 characters</li><li>mean: 133.6 characters</li><li>max: 733 characters</li></ul> | <ul><li>min: 23 characters</li><li>mean: 129.4 characters</li><li>max: 639 characters</li></ul> |
260
+ * Samples:
261
+ | anchor | positive |
262
+ |:-------------------------------------------------------|:-------------------------------------------------------------------------|
263
+ | <code>name current finance minister of pakistan</code> | <code>identify the present finance minister of pakistan</code> |
264
+ | <code>name current finance minister of pakistan</code> | <code>who is pakistan's finance minister right now</code> |
265
+ | <code>name current finance minister of pakistan</code> | <code>state the name of the current pakistani minister of finance</code> |
266
+ * Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
267
+ ```json
268
+ {
269
+ "loss": "MultipleNegativesRankingLoss",
270
+ "matryoshka_dims": [
271
+ 1024,
272
+ 768,
273
+ 512,
274
+ 256,
275
+ 128,
276
+ 64,
277
+ 32
278
+ ],
279
+ "matryoshka_weights": [
280
+ 1,
281
+ 1,
282
+ 1,
283
+ 1,
284
+ 1,
285
+ 1,
286
+ 1
287
+ ],
288
+ "n_dims_per_step": -1
289
+ }
290
+ ```
291
+ </details>
292
+ <details><summary>mjbommar/ogbert-v1-contrastive</summary>
293
+
294
+ #### mjbommar/ogbert-v1-contrastive
295
+
296
+ * Dataset: mjbommar/ogbert-v1-contrastive
297
+ * Size: 2,673,774 training samples
298
+ * Columns: <code>anchor</code> and <code>positives</code>
299
+ * Approximate statistics based on the first 100 samples:
300
+ | | anchor | positives |
301
+ |:---------|:-----------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------|
302
+ | type | string | string |
303
+ | modality | text | text |
304
+ | details | <ul><li>min: 4 characters</li><li>mean: 50.25 characters</li><li>max: 178 characters</li></ul> | <ul><li>min: 4 characters</li><li>mean: 36.68 characters</li><li>max: 121 characters</li></ul> |
305
+ * Samples:
306
+ | anchor | positives |
307
+ |:-------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
308
+ | <code>The arrangement of halftone dots or grid that encodes tonal values in a halftone image.</code> | <code>The halftone dot grid encodes tonal levels.</code> |
309
+ | <code>A formal subset of managers designated to handle day-to-day operations and implement board decisions.</code> | <code>The executive team meets monthly to review progress and adjust plans.</code> |
310
+ | <code>niche market segment</code> | <code>submarket</code> |
311
+ * Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
312
+ ```json
313
+ {
314
+ "loss": "MultipleNegativesRankingLoss",
315
+ "matryoshka_dims": [
316
+ 1024,
317
+ 768,
318
+ 512,
319
+ 256,
320
+ 128,
321
+ 64,
322
+ 32
323
+ ],
324
+ "matryoshka_weights": [
325
+ 1,
326
+ 1,
327
+ 1,
328
+ 1,
329
+ 1,
330
+ 1,
331
+ 1
332
+ ],
333
+ "n_dims_per_step": -1
334
+ }
335
+ ```
336
+ </details>
337
+ <details><summary>mjbommar/opengloss-v1.3-contrastive-examples</summary>
338
+
339
+ #### mjbommar/opengloss-v1.3-contrastive-examples
340
+
341
+ * Dataset: mjbommar/opengloss-v1.3-contrastive-examples
342
+ * Size: 161,496 training samples
343
+ * Columns: <code>anchor</code>, <code>positives</code>, <code>negatives_1</code>, and <code>negatives_2</code>
344
+ * Approximate statistics based on the first 100 samples:
345
+ | | anchor | positives | negatives_1 | negatives_2 |
346
+ |:---------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------|
347
+ | type | string | string | string | string |
348
+ | modality | text | text | text | text |
349
+ | details | <ul><li>min: 37 characters</li><li>mean: 70.41 characters</li><li>max: 145 characters</li></ul> | <ul><li>min: 36 characters</li><li>mean: 71.77 characters</li><li>max: 146 characters</li></ul> | <ul><li>min: 33 characters</li><li>mean: 72.34 characters</li><li>max: 145 characters</li></ul> | <ul><li>min: 35 characters</li><li>mean: 71.88 characters</li><li>max: 146 characters</li></ul> |
350
+ * Samples:
351
+ | anchor | positives | negatives_1 | negatives_2 |
352
+ |:---------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------|
353
+ | <code>Field interviews revealed a stony demeanor even when pressed for clarification.</code> | <code>Field interviews revealed a stoic demeanor even when pressed for clarification.</code> | <code>Field interviews revealed a warm demeanor even when pressed for clarification.</code> | <code>Field interviews revealed an empathetic demeanor even when pressed for clarification.</code> |
354
+ | <code>The board followed formal protocol during the review.</code> | <code>The board followed official protocol during the review.</code> | <code>The board followed casual protocol during the review.</code> | <code>The board followed informal protocol during the review.</code> |
355
+ | <code>The athlete carefully inspected the luge before the final run.</code> | <code>The athlete carefully inspected the toboggan before the final run.</code> | <code>The athlete carefully inspected the sled before the final run.</code> | <code>The athlete carefully inspected the bobsled before the final run.</code> |
356
+ * Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
357
+ ```json
358
+ {
359
+ "loss": "MultipleNegativesRankingLoss",
360
+ "matryoshka_dims": [
361
+ 1024,
362
+ 768,
363
+ 512,
364
+ 256,
365
+ 128,
366
+ 64,
367
+ 32
368
+ ],
369
+ "matryoshka_weights": [
370
+ 1,
371
+ 1,
372
+ 1,
373
+ 1,
374
+ 1,
375
+ 1,
376
+ 1
377
+ ],
378
+ "n_dims_per_step": -1
379
+ }
380
+ ```
381
+ </details>
382
+
383
+ ### Training Hyperparameters
384
+ #### Non-Default Hyperparameters
385
+
386
+ - `per_device_train_batch_size`: 4096
387
+ - `num_train_epochs`: 20
388
+ - `learning_rate`: 0.0001
389
+ - `warmup_steps`: 0.1
390
+ - `bf16`: True
391
+ - `batch_sampler`: no_duplicates
392
+
393
+ #### All Hyperparameters
394
+ <details><summary>Click to expand</summary>
395
+
396
+ - `per_device_train_batch_size`: 4096
397
+ - `num_train_epochs`: 20
398
+ - `max_steps`: -1
399
+ - `learning_rate`: 0.0001
400
+ - `lr_scheduler_type`: linear
401
+ - `lr_scheduler_kwargs`: None
402
+ - `warmup_steps`: 0.1
403
+ - `optim`: adamw_torch_fused
404
+ - `optim_args`: None
405
+ - `weight_decay`: 0.0
406
+ - `adam_beta1`: 0.9
407
+ - `adam_beta2`: 0.999
408
+ - `adam_epsilon`: 1e-08
409
+ - `optim_target_modules`: None
410
+ - `gradient_accumulation_steps`: 1
411
+ - `average_tokens_across_devices`: True
412
+ - `max_grad_norm`: 1.0
413
+ - `label_smoothing_factor`: 0.0
414
+ - `bf16`: True
415
+ - `fp16`: False
416
+ - `bf16_full_eval`: False
417
+ - `fp16_full_eval`: False
418
+ - `tf32`: None
419
+ - `gradient_checkpointing`: False
420
+ - `gradient_checkpointing_kwargs`: None
421
+ - `torch_compile`: False
422
+ - `torch_compile_backend`: None
423
+ - `torch_compile_mode`: None
424
+ - `use_liger_kernel`: False
425
+ - `liger_kernel_config`: None
426
+ - `use_cache`: False
427
+ - `neftune_noise_alpha`: None
428
+ - `torch_empty_cache_steps`: None
429
+ - `auto_find_batch_size`: False
430
+ - `log_on_each_node`: True
431
+ - `logging_nan_inf_filter`: True
432
+ - `include_num_input_tokens_seen`: no
433
+ - `log_level`: passive
434
+ - `log_level_replica`: warning
435
+ - `disable_tqdm`: False
436
+ - `project`: huggingface
437
+ - `trackio_space_id`: None
438
+ - `trackio_bucket_id`: None
439
+ - `trackio_static_space_id`: None
440
+ - `per_device_eval_batch_size`: 8
441
+ - `prediction_loss_only`: True
442
+ - `eval_on_start`: False
443
+ - `eval_do_concat_batches`: True
444
+ - `eval_use_gather_object`: False
445
+ - `eval_accumulation_steps`: None
446
+ - `include_for_metrics`: []
447
+ - `batch_eval_metrics`: False
448
+ - `save_only_model`: False
449
+ - `save_on_each_node`: False
450
+ - `enable_jit_checkpoint`: False
451
+ - `push_to_hub`: False
452
+ - `hub_private_repo`: None
453
+ - `hub_model_id`: None
454
+ - `hub_strategy`: every_save
455
+ - `hub_always_push`: False
456
+ - `hub_revision`: None
457
+ - `load_best_model_at_end`: False
458
+ - `ignore_data_skip`: False
459
+ - `restore_callback_states_from_checkpoint`: False
460
+ - `full_determinism`: False
461
+ - `seed`: 42
462
+ - `data_seed`: None
463
+ - `use_cpu`: False
464
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
465
+ - `parallelism_config`: None
466
+ - `dataloader_drop_last`: False
467
+ - `dataloader_num_workers`: 0
468
+ - `dataloader_pin_memory`: True
469
+ - `dataloader_persistent_workers`: False
470
+ - `dataloader_prefetch_factor`: None
471
+ - `remove_unused_columns`: True
472
+ - `label_names`: None
473
+ - `train_sampling_strategy`: random
474
+ - `length_column_name`: length
475
+ - `ddp_find_unused_parameters`: None
476
+ - `ddp_bucket_cap_mb`: None
477
+ - `ddp_broadcast_buffers`: False
478
+ - `ddp_static_graph`: None
479
+ - `ddp_backend`: None
480
+ - `ddp_timeout`: 1800
481
+ - `fsdp`: None
482
+ - `fsdp_config`: None
483
+ - `deepspeed`: None
484
+ - `debug`: []
485
+ - `skip_memory_metrics`: True
486
+ - `do_predict`: False
487
+ - `resume_from_checkpoint`: None
488
+ - `warmup_ratio`: None
489
+ - `local_rank`: -1
490
+ - `prompts`: None
491
+ - `batch_sampler`: no_duplicates
492
+ - `multi_dataset_batch_sampler`: proportional
493
+ - `router_mapping`: {}
494
+ - `learning_rate_mapping`: {}
495
+
496
+ </details>
497
+
498
+ ### Training Logs
499
+ | Epoch | Step | Training Loss |
500
+ |:-------:|:-----:|:-------------:|
501
+ | 0.0009 | 1 | 61.4605 |
502
+ | 0.9346 | 1000 | 45.8318 |
503
+ | 0.0009 | 1 | 60.6387 |
504
+ | 0.9346 | 1000 | 45.0978 |
505
+ | 1.8692 | 2000 | 44.8278 |
506
+ | 2.8037 | 3000 | 43.0278 |
507
+ | 3.7383 | 4000 | 41.6984 |
508
+ | 0.0009 | 1 | 55.3998 |
509
+ | 4.6729 | 5000 | 40.7913 |
510
+ | 5.6075 | 6000 | 40.1543 |
511
+ | 6.5421 | 7000 | 38.1761 |
512
+ | 7.4766 | 8000 | 38.8003 |
513
+ | 8.4112 | 9000 | 38.2068 |
514
+ | 9.3458 | 10000 | 37.4088 |
515
+ | 10.2804 | 11000 | 36.5805 |
516
+ | 11.2150 | 12000 | 37.1526 |
517
+ | 12.1495 | 13000 | 36.0028 |
518
+ | 13.0841 | 14000 | 35.2225 |
519
+ | 14.0187 | 15000 | 36.2388 |
520
+ | 14.9533 | 16000 | 35.1998 |
521
+ | 15.8879 | 17000 | 35.2619 |
522
+ | 16.8224 | 18000 | 35.1943 |
523
+ | 17.7570 | 19000 | 36.0396 |
524
+ | 18.6916 | 20000 | 34.2574 |
525
+ | 19.6262 | 21000 | 35.1139 |
526
+
527
+
528
+ ### Training Time
529
+ - **Training**: 2.1 hours
530
+
531
+ ### Framework Versions
532
+ - Python: 3.13.11
533
+ - Sentence Transformers: 5.6.1
534
+ - Transformers: 5.14.1
535
+ - PyTorch: 2.13.0+cu130
536
+ - Accelerate: 1.14.0
537
+ - Datasets: 5.0.0
538
+ - Tokenizers: 0.22.2
539
+
540
+ ## Citation
541
+
542
+ ### BibTeX
543
+
544
+ #### Sentence Transformers
545
+ ```bibtex
546
+ @inproceedings{reimers-2019-sentence-bert,
547
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
548
+ author = "Reimers, Nils and Gurevych, Iryna",
549
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
550
+ month = "11",
551
+ year = "2019",
552
+ publisher = "Association for Computational Linguistics",
553
+ url = "https://arxiv.org/abs/1908.10084",
554
+ }
555
+ ```
556
+
557
+ #### MatryoshkaLoss
558
+ ```bibtex
559
+ @misc{kusupati2024matryoshka,
560
+ title={Matryoshka Representation Learning},
561
+ author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
562
+ year={2024},
563
+ eprint={2205.13147},
564
+ archivePrefix={arXiv},
565
+ primaryClass={cs.LG}
566
+ }
567
+ ```
568
+
569
+ #### MultipleNegativesRankingLoss
570
+ ```bibtex
571
+ @misc{oord2019representationlearningcontrastivepredictive,
572
+ title={Representation Learning with Contrastive Predictive Coding},
573
+ author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
574
+ year={2019},
575
+ eprint={1807.03748},
576
+ archivePrefix={arXiv},
577
+ primaryClass={cs.LG},
578
+ url={https://arxiv.org/abs/1807.03748},
579
+ }
580
+ ```
581
+
582
+ <!--
583
+ ## Glossary
584
+
585
+ *Clearly define terms in order to be accessible across audiences.*
586
+ -->
587
+
588
+ <!--
589
+ ## Model Card Authors
590
+
591
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
592
+ -->
593
+
594
+ <!--
595
+ ## Model Card Contact
596
+
597
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
598
+ -->
config_sentence_transformers.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "pytorch": "2.13.0+cu130",
4
+ "sentence_transformers": "5.6.1",
5
+ "transformers": "5.14.1"
6
+ },
7
+ "default_prompt_name": null,
8
+ "model_type": "SentenceTransformer",
9
+ "prompts": {
10
+ "document": "",
11
+ "query": ""
12
+ },
13
+ "similarity_fn_name": "cosine"
14
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:10780338fbc2ed9c49998b01a7d7b5276a36a5852adebf9fa92320c50174b1b4
3
+ size 411041888
modules.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.sentence_transformer.modules.static_embedding.StaticEmbedding"
7
+ }
8
+ ]
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff