Sentence Similarity
sentence-transformers
Safetensors
English
feature-extraction
Generated from Trainer
dataset_size:4373977
loss:MatryoshkaLoss
loss:MultipleNegativesRankingLoss
Eval Results (legacy)
Instructions to use LocalWisdom/PurpleStatic with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use LocalWisdom/PurpleStatic with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("LocalWisdom/PurpleStatic") sentences = [ "crack", "master", "Academic journals often publish both print editions and digital versions simultaneously.", "ceramic ware" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
Add new SentenceTransformer model
Browse files- README.md +598 -0
- config_sentence_transformers.json +14 -0
- model.safetensors +3 -0
- modules.json +8 -0
- tokenizer.json +0 -0
README.md
ADDED
|
@@ -0,0 +1,598 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
license: wtfpl
|
| 5 |
+
tags:
|
| 6 |
+
- sentence-transformers
|
| 7 |
+
- sentence-similarity
|
| 8 |
+
- feature-extraction
|
| 9 |
+
- generated_from_trainer
|
| 10 |
+
- dataset_size:4373977
|
| 11 |
+
- loss:MatryoshkaLoss
|
| 12 |
+
- loss:MultipleNegativesRankingLoss
|
| 13 |
+
widget:
|
| 14 |
+
- source_sentence: crack
|
| 15 |
+
sentences:
|
| 16 |
+
- master
|
| 17 |
+
- Academic journals often publish both print editions and digital versions simultaneously.
|
| 18 |
+
- ceramic ware
|
| 19 |
+
- source_sentence: An area where a specific ethnic group maintains distinctive social
|
| 20 |
+
and cultural institutions within the larger city.
|
| 21 |
+
sentences:
|
| 22 |
+
- foot
|
| 23 |
+
- Policy makers study ethnic enclaves to understand immigrant entrepreneurship.
|
| 24 |
+
- economic entity
|
| 25 |
+
- source_sentence: We lease various facilities, office equipment, plant equipment
|
| 26 |
+
and railcars under operating leases.
|
| 27 |
+
sentences:
|
| 28 |
+
- Under the ESPP, qualified employees are permitted to invest as much as ten percent
|
| 29 |
+
of their base pay to acquire company common stock at a discount of 15 percent
|
| 30 |
+
off the closing market price on the day of purchase.
|
| 31 |
+
- On March 1, 2006, the total number of full-time staff members across ACGL and
|
| 32 |
+
its subsidiary companies was roughly 980.
|
| 33 |
+
- Our company utilizes operating leases for the rental of various types of equipment,
|
| 34 |
+
including railcars, plant machinery, office gear, and various facilities.
|
| 35 |
+
- source_sentence: How will the ban of Rs 500 and Rs 1000 notes affect Indian economy?
|
| 36 |
+
sentences:
|
| 37 |
+
- Where can I get professional photo booths service in Sydney?
|
| 38 |
+
- What are your views on India banning 500 and 1000 notes? In what way it will affect
|
| 39 |
+
Indian economy?
|
| 40 |
+
- What are some of your favorite sayings?
|
| 41 |
+
- source_sentence: How do you write a song?
|
| 42 |
+
sentences:
|
| 43 |
+
- How did Portugal become an independent country from Spain?
|
| 44 |
+
- Who created the "GOD"?
|
| 45 |
+
- How do I write a song?
|
| 46 |
+
pipeline_tag: sentence-similarity
|
| 47 |
+
library_name: sentence-transformers
|
| 48 |
+
---
|
| 49 |
+
|
| 50 |
+
# PurpleStatic: Static Embeddings
|
| 51 |
+
|
| 52 |
+
This is a [sentence-transformers](https://www.SBERT.net) model trained on the BatuhanECB/FinModernBERT-pairs-sec-synthetic-v1, Heliosoph/Quora-Question-Pairs, owenkaplinsky/wildchat-paraphrases, mjbommar/ogbert-v1-contrastive and mjbommar/opengloss-v1.3-contrastive-examples datasets. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for retrieval.
|
| 53 |
+
|
| 54 |
+
## Model Details
|
| 55 |
+
|
| 56 |
+
### Model Description
|
| 57 |
+
- **Model Type:** Sentence Transformer
|
| 58 |
+
<!-- - **Base model:** [Unknown](https://huggingface.co/unknown) -->
|
| 59 |
+
- **Maximum Sequence Length:** inf tokens
|
| 60 |
+
- **Output Dimensionality:** 1024 dimensions
|
| 61 |
+
- **Similarity Function:** Cosine Similarity
|
| 62 |
+
- **Supported Modality:** Text
|
| 63 |
+
- **Training Datasets:**
|
| 64 |
+
- BatuhanECB/FinModernBERT-pairs-sec-synthetic-v1
|
| 65 |
+
- Heliosoph/Quora-Question-Pairs
|
| 66 |
+
- owenkaplinsky/wildchat-paraphrases
|
| 67 |
+
- mjbommar/ogbert-v1-contrastive
|
| 68 |
+
- mjbommar/opengloss-v1.3-contrastive-examples
|
| 69 |
+
- **Language:** en
|
| 70 |
+
- **License:** wtfpl
|
| 71 |
+
|
| 72 |
+
### Model Sources
|
| 73 |
+
|
| 74 |
+
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
|
| 75 |
+
- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)
|
| 76 |
+
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
|
| 77 |
+
|
| 78 |
+
### Full Model Architecture
|
| 79 |
+
|
| 80 |
+
```
|
| 81 |
+
SentenceTransformer(
|
| 82 |
+
(0): StaticEmbedding({})
|
| 83 |
+
)
|
| 84 |
+
```
|
| 85 |
+
|
| 86 |
+
## Usage
|
| 87 |
+
|
| 88 |
+
### Direct Usage (Sentence Transformers)
|
| 89 |
+
|
| 90 |
+
First install the Sentence Transformers library:
|
| 91 |
+
|
| 92 |
+
```bash
|
| 93 |
+
pip install -U sentence-transformers
|
| 94 |
+
```
|
| 95 |
+
Then you can load this model and run inference.
|
| 96 |
+
```python
|
| 97 |
+
from sentence_transformers import SentenceTransformer
|
| 98 |
+
|
| 99 |
+
# Download from the 🤗 Hub
|
| 100 |
+
model = SentenceTransformer("LocalWisdom/PurpleStatic")
|
| 101 |
+
# Run inference
|
| 102 |
+
sentences = [
|
| 103 |
+
'How do you write a song?',
|
| 104 |
+
'How do I write a song?',
|
| 105 |
+
'How did Portugal become an independent country from Spain?',
|
| 106 |
+
]
|
| 107 |
+
embeddings = model.encode(sentences)
|
| 108 |
+
print(embeddings.shape)
|
| 109 |
+
# [3, 1024]
|
| 110 |
+
|
| 111 |
+
# Get the similarity scores for the embeddings
|
| 112 |
+
similarities = model.similarity(embeddings, embeddings)
|
| 113 |
+
print(similarities)
|
| 114 |
+
# tensor([[1.0000, 0.8566, 0.1409],
|
| 115 |
+
# [0.8566, 1.0000, 0.1259],
|
| 116 |
+
# [0.1409, 0.1259, 1.0000]])
|
| 117 |
+
```
|
| 118 |
+
<!--
|
| 119 |
+
### Direct Usage (Transformers)
|
| 120 |
+
|
| 121 |
+
<details><summary>Click to see the direct usage in Transformers</summary>
|
| 122 |
+
|
| 123 |
+
</details>
|
| 124 |
+
-->
|
| 125 |
+
|
| 126 |
+
<!--
|
| 127 |
+
### Downstream Usage (Sentence Transformers)
|
| 128 |
+
|
| 129 |
+
You can finetune this model on your own dataset.
|
| 130 |
+
|
| 131 |
+
<details><summary>Click to expand</summary>
|
| 132 |
+
|
| 133 |
+
</details>
|
| 134 |
+
-->
|
| 135 |
+
|
| 136 |
+
<!--
|
| 137 |
+
### Out-of-Scope Use
|
| 138 |
+
|
| 139 |
+
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
|
| 140 |
+
-->
|
| 141 |
+
|
| 142 |
+
<!--
|
| 143 |
+
## Bias, Risks and Limitations
|
| 144 |
+
|
| 145 |
+
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
|
| 146 |
+
-->
|
| 147 |
+
|
| 148 |
+
<!--
|
| 149 |
+
### Recommendations
|
| 150 |
+
|
| 151 |
+
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
|
| 152 |
+
-->
|
| 153 |
+
|
| 154 |
+
## Training Details
|
| 155 |
+
|
| 156 |
+
### Training Datasets
|
| 157 |
+
<details><summary>BatuhanECB/FinModernBERT-pairs-sec-synthetic-v1</summary>
|
| 158 |
+
|
| 159 |
+
#### BatuhanECB/FinModernBERT-pairs-sec-synthetic-v1
|
| 160 |
+
|
| 161 |
+
* Dataset: BatuhanECB/FinModernBERT-pairs-sec-synthetic-v1
|
| 162 |
+
* Size: 5,694 training samples
|
| 163 |
+
* Columns: <code>anchor</code> and <code>positive</code>
|
| 164 |
+
* Approximate statistics based on the first 100 samples:
|
| 165 |
+
| | anchor | positive |
|
| 166 |
+
|:---------|:-------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------|
|
| 167 |
+
| type | string | string |
|
| 168 |
+
| modality | text | text |
|
| 169 |
+
| details | <ul><li>min: 47 characters</li><li>mean: 135.34 characters</li><li>max: 352 characters</li></ul> | <ul><li>min: 70 characters</li><li>mean: 146.17 characters</li><li>max: 322 characters</li></ul> |
|
| 170 |
+
* Samples:
|
| 171 |
+
| anchor | positive |
|
| 172 |
+
|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
| 173 |
+
| <code>Of our total net revenue of $10.8 billion in the fiscal year ended October 31, 2000, we generated 44.2% in the United States and 55.8% internationally.</code> | <code>In the fiscal year concluding October 31, 2000, the company produced 44.2% of its $10.8 billion total net revenue within the United States and 55.8% from international markets.</code> |
|
| 174 |
+
| <code>Our sales strategy is to sell to and service our largest accounts (hospital and corporate business) directly while employing third-party distributors and manufacturer's representatives for smaller or more geographically dispersed countries.</code> | <code>We utilize a sales approach where we directly manage and sell to our major hospital and corporate clients, while relying on external distributors and manufacturer agents to reach smaller markets or those in remote locations.</code> |
|
| 175 |
+
| <code>In August 2000, we announced a restructuring of our healthcare solutions business.</code> | <code>The company declared a reorganization of its healthcare solutions division in August 2000.</code> |
|
| 176 |
+
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
|
| 177 |
+
```json
|
| 178 |
+
{
|
| 179 |
+
"loss": "MultipleNegativesRankingLoss",
|
| 180 |
+
"matryoshka_dims": [
|
| 181 |
+
1024,
|
| 182 |
+
768,
|
| 183 |
+
512,
|
| 184 |
+
256,
|
| 185 |
+
128,
|
| 186 |
+
64,
|
| 187 |
+
32
|
| 188 |
+
],
|
| 189 |
+
"matryoshka_weights": [
|
| 190 |
+
1,
|
| 191 |
+
1,
|
| 192 |
+
1,
|
| 193 |
+
1,
|
| 194 |
+
1,
|
| 195 |
+
1,
|
| 196 |
+
1
|
| 197 |
+
],
|
| 198 |
+
"n_dims_per_step": -1
|
| 199 |
+
}
|
| 200 |
+
```
|
| 201 |
+
</details>
|
| 202 |
+
<details><summary>Heliosoph/Quora-Question-Pairs</summary>
|
| 203 |
+
|
| 204 |
+
#### Heliosoph/Quora-Question-Pairs
|
| 205 |
+
|
| 206 |
+
* Dataset: Heliosoph/Quora-Question-Pairs
|
| 207 |
+
* Size: 149,263 training samples
|
| 208 |
+
* Columns: <code>anchor</code> and <code>positive</code>
|
| 209 |
+
* Approximate statistics based on the first 100 samples:
|
| 210 |
+
| | anchor | positive |
|
| 211 |
+
|:---------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------|
|
| 212 |
+
| type | string | string |
|
| 213 |
+
| modality | text | text |
|
| 214 |
+
| details | <ul><li>min: 16 characters</li><li>mean: 54.75 characters</li><li>max: 139 characters</li></ul> | <ul><li>min: 21 characters</li><li>mean: 54.34 characters</li><li>max: 127 characters</li></ul> |
|
| 215 |
+
* Samples:
|
| 216 |
+
| anchor | positive |
|
| 217 |
+
|:----------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------|
|
| 218 |
+
| <code>Astrology: I am a Capricorn Sun Cap moon and cap rising...what does that say about me?</code> | <code>I'm a triple Capricorn (Sun, Moon and ascendant in Capricorn) What does this say about me?</code> |
|
| 219 |
+
| <code>How can I be a good geologist?</code> | <code>What should I do to be a great geologist?</code> |
|
| 220 |
+
| <code>How do I read and find my YouTube comments?</code> | <code>How can I see all my Youtube comments?</code> |
|
| 221 |
+
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
|
| 222 |
+
```json
|
| 223 |
+
{
|
| 224 |
+
"loss": "MultipleNegativesRankingLoss",
|
| 225 |
+
"matryoshka_dims": [
|
| 226 |
+
1024,
|
| 227 |
+
768,
|
| 228 |
+
512,
|
| 229 |
+
256,
|
| 230 |
+
128,
|
| 231 |
+
64,
|
| 232 |
+
32
|
| 233 |
+
],
|
| 234 |
+
"matryoshka_weights": [
|
| 235 |
+
1,
|
| 236 |
+
1,
|
| 237 |
+
1,
|
| 238 |
+
1,
|
| 239 |
+
1,
|
| 240 |
+
1,
|
| 241 |
+
1
|
| 242 |
+
],
|
| 243 |
+
"n_dims_per_step": -1
|
| 244 |
+
}
|
| 245 |
+
```
|
| 246 |
+
</details>
|
| 247 |
+
<details><summary>owenkaplinsky/wildchat-paraphrases</summary>
|
| 248 |
+
|
| 249 |
+
#### owenkaplinsky/wildchat-paraphrases
|
| 250 |
+
|
| 251 |
+
* Dataset: owenkaplinsky/wildchat-paraphrases
|
| 252 |
+
* Size: 1,383,750 training samples
|
| 253 |
+
* Columns: <code>anchor</code> and <code>positive</code>
|
| 254 |
+
* Approximate statistics based on the first 100 samples:
|
| 255 |
+
| | anchor | positive |
|
| 256 |
+
|:---------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------|
|
| 257 |
+
| type | string | string |
|
| 258 |
+
| modality | text | text |
|
| 259 |
+
| details | <ul><li>min: 20 characters</li><li>mean: 133.6 characters</li><li>max: 733 characters</li></ul> | <ul><li>min: 23 characters</li><li>mean: 129.4 characters</li><li>max: 639 characters</li></ul> |
|
| 260 |
+
* Samples:
|
| 261 |
+
| anchor | positive |
|
| 262 |
+
|:-------------------------------------------------------|:-------------------------------------------------------------------------|
|
| 263 |
+
| <code>name current finance minister of pakistan</code> | <code>identify the present finance minister of pakistan</code> |
|
| 264 |
+
| <code>name current finance minister of pakistan</code> | <code>who is pakistan's finance minister right now</code> |
|
| 265 |
+
| <code>name current finance minister of pakistan</code> | <code>state the name of the current pakistani minister of finance</code> |
|
| 266 |
+
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
|
| 267 |
+
```json
|
| 268 |
+
{
|
| 269 |
+
"loss": "MultipleNegativesRankingLoss",
|
| 270 |
+
"matryoshka_dims": [
|
| 271 |
+
1024,
|
| 272 |
+
768,
|
| 273 |
+
512,
|
| 274 |
+
256,
|
| 275 |
+
128,
|
| 276 |
+
64,
|
| 277 |
+
32
|
| 278 |
+
],
|
| 279 |
+
"matryoshka_weights": [
|
| 280 |
+
1,
|
| 281 |
+
1,
|
| 282 |
+
1,
|
| 283 |
+
1,
|
| 284 |
+
1,
|
| 285 |
+
1,
|
| 286 |
+
1
|
| 287 |
+
],
|
| 288 |
+
"n_dims_per_step": -1
|
| 289 |
+
}
|
| 290 |
+
```
|
| 291 |
+
</details>
|
| 292 |
+
<details><summary>mjbommar/ogbert-v1-contrastive</summary>
|
| 293 |
+
|
| 294 |
+
#### mjbommar/ogbert-v1-contrastive
|
| 295 |
+
|
| 296 |
+
* Dataset: mjbommar/ogbert-v1-contrastive
|
| 297 |
+
* Size: 2,673,774 training samples
|
| 298 |
+
* Columns: <code>anchor</code> and <code>positives</code>
|
| 299 |
+
* Approximate statistics based on the first 100 samples:
|
| 300 |
+
| | anchor | positives |
|
| 301 |
+
|:---------|:-----------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------|
|
| 302 |
+
| type | string | string |
|
| 303 |
+
| modality | text | text |
|
| 304 |
+
| details | <ul><li>min: 4 characters</li><li>mean: 50.25 characters</li><li>max: 178 characters</li></ul> | <ul><li>min: 4 characters</li><li>mean: 36.68 characters</li><li>max: 121 characters</li></ul> |
|
| 305 |
+
* Samples:
|
| 306 |
+
| anchor | positives |
|
| 307 |
+
|:-------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
|
| 308 |
+
| <code>The arrangement of halftone dots or grid that encodes tonal values in a halftone image.</code> | <code>The halftone dot grid encodes tonal levels.</code> |
|
| 309 |
+
| <code>A formal subset of managers designated to handle day-to-day operations and implement board decisions.</code> | <code>The executive team meets monthly to review progress and adjust plans.</code> |
|
| 310 |
+
| <code>niche market segment</code> | <code>submarket</code> |
|
| 311 |
+
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
|
| 312 |
+
```json
|
| 313 |
+
{
|
| 314 |
+
"loss": "MultipleNegativesRankingLoss",
|
| 315 |
+
"matryoshka_dims": [
|
| 316 |
+
1024,
|
| 317 |
+
768,
|
| 318 |
+
512,
|
| 319 |
+
256,
|
| 320 |
+
128,
|
| 321 |
+
64,
|
| 322 |
+
32
|
| 323 |
+
],
|
| 324 |
+
"matryoshka_weights": [
|
| 325 |
+
1,
|
| 326 |
+
1,
|
| 327 |
+
1,
|
| 328 |
+
1,
|
| 329 |
+
1,
|
| 330 |
+
1,
|
| 331 |
+
1
|
| 332 |
+
],
|
| 333 |
+
"n_dims_per_step": -1
|
| 334 |
+
}
|
| 335 |
+
```
|
| 336 |
+
</details>
|
| 337 |
+
<details><summary>mjbommar/opengloss-v1.3-contrastive-examples</summary>
|
| 338 |
+
|
| 339 |
+
#### mjbommar/opengloss-v1.3-contrastive-examples
|
| 340 |
+
|
| 341 |
+
* Dataset: mjbommar/opengloss-v1.3-contrastive-examples
|
| 342 |
+
* Size: 161,496 training samples
|
| 343 |
+
* Columns: <code>anchor</code>, <code>positives</code>, <code>negatives_1</code>, and <code>negatives_2</code>
|
| 344 |
+
* Approximate statistics based on the first 100 samples:
|
| 345 |
+
| | anchor | positives | negatives_1 | negatives_2 |
|
| 346 |
+
|:---------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------|
|
| 347 |
+
| type | string | string | string | string |
|
| 348 |
+
| modality | text | text | text | text |
|
| 349 |
+
| details | <ul><li>min: 37 characters</li><li>mean: 70.41 characters</li><li>max: 145 characters</li></ul> | <ul><li>min: 36 characters</li><li>mean: 71.77 characters</li><li>max: 146 characters</li></ul> | <ul><li>min: 33 characters</li><li>mean: 72.34 characters</li><li>max: 145 characters</li></ul> | <ul><li>min: 35 characters</li><li>mean: 71.88 characters</li><li>max: 146 characters</li></ul> |
|
| 350 |
+
* Samples:
|
| 351 |
+
| anchor | positives | negatives_1 | negatives_2 |
|
| 352 |
+
|:---------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------|
|
| 353 |
+
| <code>Field interviews revealed a stony demeanor even when pressed for clarification.</code> | <code>Field interviews revealed a stoic demeanor even when pressed for clarification.</code> | <code>Field interviews revealed a warm demeanor even when pressed for clarification.</code> | <code>Field interviews revealed an empathetic demeanor even when pressed for clarification.</code> |
|
| 354 |
+
| <code>The board followed formal protocol during the review.</code> | <code>The board followed official protocol during the review.</code> | <code>The board followed casual protocol during the review.</code> | <code>The board followed informal protocol during the review.</code> |
|
| 355 |
+
| <code>The athlete carefully inspected the luge before the final run.</code> | <code>The athlete carefully inspected the toboggan before the final run.</code> | <code>The athlete carefully inspected the sled before the final run.</code> | <code>The athlete carefully inspected the bobsled before the final run.</code> |
|
| 356 |
+
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
|
| 357 |
+
```json
|
| 358 |
+
{
|
| 359 |
+
"loss": "MultipleNegativesRankingLoss",
|
| 360 |
+
"matryoshka_dims": [
|
| 361 |
+
1024,
|
| 362 |
+
768,
|
| 363 |
+
512,
|
| 364 |
+
256,
|
| 365 |
+
128,
|
| 366 |
+
64,
|
| 367 |
+
32
|
| 368 |
+
],
|
| 369 |
+
"matryoshka_weights": [
|
| 370 |
+
1,
|
| 371 |
+
1,
|
| 372 |
+
1,
|
| 373 |
+
1,
|
| 374 |
+
1,
|
| 375 |
+
1,
|
| 376 |
+
1
|
| 377 |
+
],
|
| 378 |
+
"n_dims_per_step": -1
|
| 379 |
+
}
|
| 380 |
+
```
|
| 381 |
+
</details>
|
| 382 |
+
|
| 383 |
+
### Training Hyperparameters
|
| 384 |
+
#### Non-Default Hyperparameters
|
| 385 |
+
|
| 386 |
+
- `per_device_train_batch_size`: 4096
|
| 387 |
+
- `num_train_epochs`: 20
|
| 388 |
+
- `learning_rate`: 0.0001
|
| 389 |
+
- `warmup_steps`: 0.1
|
| 390 |
+
- `bf16`: True
|
| 391 |
+
- `batch_sampler`: no_duplicates
|
| 392 |
+
|
| 393 |
+
#### All Hyperparameters
|
| 394 |
+
<details><summary>Click to expand</summary>
|
| 395 |
+
|
| 396 |
+
- `per_device_train_batch_size`: 4096
|
| 397 |
+
- `num_train_epochs`: 20
|
| 398 |
+
- `max_steps`: -1
|
| 399 |
+
- `learning_rate`: 0.0001
|
| 400 |
+
- `lr_scheduler_type`: linear
|
| 401 |
+
- `lr_scheduler_kwargs`: None
|
| 402 |
+
- `warmup_steps`: 0.1
|
| 403 |
+
- `optim`: adamw_torch_fused
|
| 404 |
+
- `optim_args`: None
|
| 405 |
+
- `weight_decay`: 0.0
|
| 406 |
+
- `adam_beta1`: 0.9
|
| 407 |
+
- `adam_beta2`: 0.999
|
| 408 |
+
- `adam_epsilon`: 1e-08
|
| 409 |
+
- `optim_target_modules`: None
|
| 410 |
+
- `gradient_accumulation_steps`: 1
|
| 411 |
+
- `average_tokens_across_devices`: True
|
| 412 |
+
- `max_grad_norm`: 1.0
|
| 413 |
+
- `label_smoothing_factor`: 0.0
|
| 414 |
+
- `bf16`: True
|
| 415 |
+
- `fp16`: False
|
| 416 |
+
- `bf16_full_eval`: False
|
| 417 |
+
- `fp16_full_eval`: False
|
| 418 |
+
- `tf32`: None
|
| 419 |
+
- `gradient_checkpointing`: False
|
| 420 |
+
- `gradient_checkpointing_kwargs`: None
|
| 421 |
+
- `torch_compile`: False
|
| 422 |
+
- `torch_compile_backend`: None
|
| 423 |
+
- `torch_compile_mode`: None
|
| 424 |
+
- `use_liger_kernel`: False
|
| 425 |
+
- `liger_kernel_config`: None
|
| 426 |
+
- `use_cache`: False
|
| 427 |
+
- `neftune_noise_alpha`: None
|
| 428 |
+
- `torch_empty_cache_steps`: None
|
| 429 |
+
- `auto_find_batch_size`: False
|
| 430 |
+
- `log_on_each_node`: True
|
| 431 |
+
- `logging_nan_inf_filter`: True
|
| 432 |
+
- `include_num_input_tokens_seen`: no
|
| 433 |
+
- `log_level`: passive
|
| 434 |
+
- `log_level_replica`: warning
|
| 435 |
+
- `disable_tqdm`: False
|
| 436 |
+
- `project`: huggingface
|
| 437 |
+
- `trackio_space_id`: None
|
| 438 |
+
- `trackio_bucket_id`: None
|
| 439 |
+
- `trackio_static_space_id`: None
|
| 440 |
+
- `per_device_eval_batch_size`: 8
|
| 441 |
+
- `prediction_loss_only`: True
|
| 442 |
+
- `eval_on_start`: False
|
| 443 |
+
- `eval_do_concat_batches`: True
|
| 444 |
+
- `eval_use_gather_object`: False
|
| 445 |
+
- `eval_accumulation_steps`: None
|
| 446 |
+
- `include_for_metrics`: []
|
| 447 |
+
- `batch_eval_metrics`: False
|
| 448 |
+
- `save_only_model`: False
|
| 449 |
+
- `save_on_each_node`: False
|
| 450 |
+
- `enable_jit_checkpoint`: False
|
| 451 |
+
- `push_to_hub`: False
|
| 452 |
+
- `hub_private_repo`: None
|
| 453 |
+
- `hub_model_id`: None
|
| 454 |
+
- `hub_strategy`: every_save
|
| 455 |
+
- `hub_always_push`: False
|
| 456 |
+
- `hub_revision`: None
|
| 457 |
+
- `load_best_model_at_end`: False
|
| 458 |
+
- `ignore_data_skip`: False
|
| 459 |
+
- `restore_callback_states_from_checkpoint`: False
|
| 460 |
+
- `full_determinism`: False
|
| 461 |
+
- `seed`: 42
|
| 462 |
+
- `data_seed`: None
|
| 463 |
+
- `use_cpu`: False
|
| 464 |
+
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
|
| 465 |
+
- `parallelism_config`: None
|
| 466 |
+
- `dataloader_drop_last`: False
|
| 467 |
+
- `dataloader_num_workers`: 0
|
| 468 |
+
- `dataloader_pin_memory`: True
|
| 469 |
+
- `dataloader_persistent_workers`: False
|
| 470 |
+
- `dataloader_prefetch_factor`: None
|
| 471 |
+
- `remove_unused_columns`: True
|
| 472 |
+
- `label_names`: None
|
| 473 |
+
- `train_sampling_strategy`: random
|
| 474 |
+
- `length_column_name`: length
|
| 475 |
+
- `ddp_find_unused_parameters`: None
|
| 476 |
+
- `ddp_bucket_cap_mb`: None
|
| 477 |
+
- `ddp_broadcast_buffers`: False
|
| 478 |
+
- `ddp_static_graph`: None
|
| 479 |
+
- `ddp_backend`: None
|
| 480 |
+
- `ddp_timeout`: 1800
|
| 481 |
+
- `fsdp`: None
|
| 482 |
+
- `fsdp_config`: None
|
| 483 |
+
- `deepspeed`: None
|
| 484 |
+
- `debug`: []
|
| 485 |
+
- `skip_memory_metrics`: True
|
| 486 |
+
- `do_predict`: False
|
| 487 |
+
- `resume_from_checkpoint`: None
|
| 488 |
+
- `warmup_ratio`: None
|
| 489 |
+
- `local_rank`: -1
|
| 490 |
+
- `prompts`: None
|
| 491 |
+
- `batch_sampler`: no_duplicates
|
| 492 |
+
- `multi_dataset_batch_sampler`: proportional
|
| 493 |
+
- `router_mapping`: {}
|
| 494 |
+
- `learning_rate_mapping`: {}
|
| 495 |
+
|
| 496 |
+
</details>
|
| 497 |
+
|
| 498 |
+
### Training Logs
|
| 499 |
+
| Epoch | Step | Training Loss |
|
| 500 |
+
|:-------:|:-----:|:-------------:|
|
| 501 |
+
| 0.0009 | 1 | 61.4605 |
|
| 502 |
+
| 0.9346 | 1000 | 45.8318 |
|
| 503 |
+
| 0.0009 | 1 | 60.6387 |
|
| 504 |
+
| 0.9346 | 1000 | 45.0978 |
|
| 505 |
+
| 1.8692 | 2000 | 44.8278 |
|
| 506 |
+
| 2.8037 | 3000 | 43.0278 |
|
| 507 |
+
| 3.7383 | 4000 | 41.6984 |
|
| 508 |
+
| 0.0009 | 1 | 55.3998 |
|
| 509 |
+
| 4.6729 | 5000 | 40.7913 |
|
| 510 |
+
| 5.6075 | 6000 | 40.1543 |
|
| 511 |
+
| 6.5421 | 7000 | 38.1761 |
|
| 512 |
+
| 7.4766 | 8000 | 38.8003 |
|
| 513 |
+
| 8.4112 | 9000 | 38.2068 |
|
| 514 |
+
| 9.3458 | 10000 | 37.4088 |
|
| 515 |
+
| 10.2804 | 11000 | 36.5805 |
|
| 516 |
+
| 11.2150 | 12000 | 37.1526 |
|
| 517 |
+
| 12.1495 | 13000 | 36.0028 |
|
| 518 |
+
| 13.0841 | 14000 | 35.2225 |
|
| 519 |
+
| 14.0187 | 15000 | 36.2388 |
|
| 520 |
+
| 14.9533 | 16000 | 35.1998 |
|
| 521 |
+
| 15.8879 | 17000 | 35.2619 |
|
| 522 |
+
| 16.8224 | 18000 | 35.1943 |
|
| 523 |
+
| 17.7570 | 19000 | 36.0396 |
|
| 524 |
+
| 18.6916 | 20000 | 34.2574 |
|
| 525 |
+
| 19.6262 | 21000 | 35.1139 |
|
| 526 |
+
|
| 527 |
+
|
| 528 |
+
### Training Time
|
| 529 |
+
- **Training**: 2.1 hours
|
| 530 |
+
|
| 531 |
+
### Framework Versions
|
| 532 |
+
- Python: 3.13.11
|
| 533 |
+
- Sentence Transformers: 5.6.1
|
| 534 |
+
- Transformers: 5.14.1
|
| 535 |
+
- PyTorch: 2.13.0+cu130
|
| 536 |
+
- Accelerate: 1.14.0
|
| 537 |
+
- Datasets: 5.0.0
|
| 538 |
+
- Tokenizers: 0.22.2
|
| 539 |
+
|
| 540 |
+
## Citation
|
| 541 |
+
|
| 542 |
+
### BibTeX
|
| 543 |
+
|
| 544 |
+
#### Sentence Transformers
|
| 545 |
+
```bibtex
|
| 546 |
+
@inproceedings{reimers-2019-sentence-bert,
|
| 547 |
+
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
|
| 548 |
+
author = "Reimers, Nils and Gurevych, Iryna",
|
| 549 |
+
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
|
| 550 |
+
month = "11",
|
| 551 |
+
year = "2019",
|
| 552 |
+
publisher = "Association for Computational Linguistics",
|
| 553 |
+
url = "https://arxiv.org/abs/1908.10084",
|
| 554 |
+
}
|
| 555 |
+
```
|
| 556 |
+
|
| 557 |
+
#### MatryoshkaLoss
|
| 558 |
+
```bibtex
|
| 559 |
+
@misc{kusupati2024matryoshka,
|
| 560 |
+
title={Matryoshka Representation Learning},
|
| 561 |
+
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
|
| 562 |
+
year={2024},
|
| 563 |
+
eprint={2205.13147},
|
| 564 |
+
archivePrefix={arXiv},
|
| 565 |
+
primaryClass={cs.LG}
|
| 566 |
+
}
|
| 567 |
+
```
|
| 568 |
+
|
| 569 |
+
#### MultipleNegativesRankingLoss
|
| 570 |
+
```bibtex
|
| 571 |
+
@misc{oord2019representationlearningcontrastivepredictive,
|
| 572 |
+
title={Representation Learning with Contrastive Predictive Coding},
|
| 573 |
+
author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
|
| 574 |
+
year={2019},
|
| 575 |
+
eprint={1807.03748},
|
| 576 |
+
archivePrefix={arXiv},
|
| 577 |
+
primaryClass={cs.LG},
|
| 578 |
+
url={https://arxiv.org/abs/1807.03748},
|
| 579 |
+
}
|
| 580 |
+
```
|
| 581 |
+
|
| 582 |
+
<!--
|
| 583 |
+
## Glossary
|
| 584 |
+
|
| 585 |
+
*Clearly define terms in order to be accessible across audiences.*
|
| 586 |
+
-->
|
| 587 |
+
|
| 588 |
+
<!--
|
| 589 |
+
## Model Card Authors
|
| 590 |
+
|
| 591 |
+
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
|
| 592 |
+
-->
|
| 593 |
+
|
| 594 |
+
<!--
|
| 595 |
+
## Model Card Contact
|
| 596 |
+
|
| 597 |
+
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
|
| 598 |
+
-->
|
config_sentence_transformers.json
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"__version__": {
|
| 3 |
+
"pytorch": "2.13.0+cu130",
|
| 4 |
+
"sentence_transformers": "5.6.1",
|
| 5 |
+
"transformers": "5.14.1"
|
| 6 |
+
},
|
| 7 |
+
"default_prompt_name": null,
|
| 8 |
+
"model_type": "SentenceTransformer",
|
| 9 |
+
"prompts": {
|
| 10 |
+
"document": "",
|
| 11 |
+
"query": ""
|
| 12 |
+
},
|
| 13 |
+
"similarity_fn_name": "cosine"
|
| 14 |
+
}
|
model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:10780338fbc2ed9c49998b01a7d7b5276a36a5852adebf9fa92320c50174b1b4
|
| 3 |
+
size 411041888
|
modules.json
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[
|
| 2 |
+
{
|
| 3 |
+
"idx": 0,
|
| 4 |
+
"name": "0",
|
| 5 |
+
"path": "",
|
| 6 |
+
"type": "sentence_transformers.sentence_transformer.modules.static_embedding.StaticEmbedding"
|
| 7 |
+
}
|
| 8 |
+
]
|
tokenizer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|