The Key of Solomon details incantations, prayers, and invocations that when said exactly right allow an adept to harness supernatural powers, including non-human intelligences. Messing up an incantation even slightly can result in disaster.
While using LLMs, I constantly add little modifiers to my main prompts to shift the model’s outputs to our preferred style. “Please be concise”, “avoid em dashes and semicolons”, “restrict inline comments to 8 words or less” and so on. These “style prompts” work pretty well, but I have found them unpredictable. Also, sometimes two “style prompts” that seem to mean the same thing to a human reader will change a model’s outputs in different ways.
Anthropic recently posted some docs recommending adding Please remove all mannered prose to avoid the LLM “slop” tone that people have learned to tune out. It seems to work pretty well, but the strangeness (and Claudeness) of that phrase struck my curiosity and made me wonder about the other style prompts that I use while working with LLMs. It seems to me that to use LLMs better we need a more rigorous treatment of this subject.
1) How specific is a style prompt’s effect to its wording? Do style prompts similar to Please remove all mannered prose change llm outputs in similar ways?
2) How consistent is the effect of a style prompt across a set of different types of tasks?
3) How do Please remove all mannered prose outputs relate to outputs from an “opposite” style prompt such as Please use mannered prose? Can we characterize other “style prompt duals” in the same way?
4) Do these style prompts have the effects we intend?
To work effectively with LLMs, we need to understand how our inputs and context shift model outputs. Engineers can build single-purpose eval sets for heavily reused tasks, but at least in my world almost all prompts are too specific, urgent, or context-dependent to stop and build an eval set. Shipping models with more thorough quantitative documentation of how prompt modifiers and added context affect generated outputs could make them better “daily drivers” in these typical use cases.
Background
If you are interested in this stuff, I recommend reading Stolfo et al., Improving Instruction-Following through Activation Steering (ICLR 2025) . I will use a modified framework from that paper for this investigation.
We will investigate these questions with three methodological tools: residual geometry a la Solfo et al. and Zou eta l. , the logit lens a la nostalgebraist , and output stylometry using standard readability metrics (Flesch 1949 , Guiraud’s 1954 book).
Approach
To do this kind of interpretability work, we need to inspect a model’s intermediate state while it is responding. That means it needs to be open-weight and small enough to work on my Apple M3 Pro w/ 18 GB of RAM. I chose gemma-2-2b-it . I would love to see this analysis run on a larger model or a Claude.
Prompting
To evaluate style prompt consistency across different tasks, we need a corpus of main prompts that we can augment with our style prompts. I chose to copy Stolfo et al. here and use the IFEval prompt set. I decided to just use the base prompts without the extra “avoid this punctuation mark” or “finish your output with this phrase” evals. To evaluate differences between style prompts that seem similar and dissimilar to humans, I needed to curate a set of style prompts. I opted to organize these into negative control (no style prompt), positive control (labeled placebo), and 11 clusters of styles. I chose the 3 prompts within each cluster with the intent to achieve the same effect on the model’s outputs, although we’ll see plenty of unexpected differences within clusters later.
We can organize these clusters into opposing directions along the same conceptual axis. For instance, avoid mannered prose and use mannered prose should have dissimilar effects on generated text.
Axis
Cluster
Style prompts
—
none (negative control)
(no style prompt)
—
placebo (positive control)
Answer the request below.
mannered
plain
Avoid mannered prose.
mannered
ornate
Use mannered prose.
length
brief
Keep it brief.
length
tokens
Minimize output tokens.
length
verbose
Be thorough and detailed.
tone
tone_formal
Use a formal tone.
tone
tone_friendly
Write in a friendly tone.
reasoning
cot
Please explain your reasoning first.
reasoning
direct
Please answer without explaining your reasoning.
careful
careful
Make no mistakes.
careful
careless
Make mistakes.
These style prompts were appended before the start of the main prompt. I kept the temperature at zero, so all sampling is deterministic and I take the argmax token at each step.
I organized the base prompts into 6 task categories based on the InstructGPT task taxonomy:
task type
n
Generation
333
Open QA
69
Closed QA
45
Rewrite
45
Brainstorming
31
Summarization
15
Inspecting the model’s internal state
Modern transformer LLMs are roughly:
1) a tokenizer (vocabulary -> tokens)
2) an embedding block (tokens -> embedding space)
3) \(n\) self-attention + MLP / FCN layer blocks, all in embedding space. \(\text{block}_i\)’s output is \(\text{block}_{i+1}\)’s input.
4) An un-embedding layer (embeddings -> tokens / vocabulary)
5) Softmax over the vocabulary to sample output tokens
A prompt input propagates through the network’s blocks sequentially. Each of those blocks outputs a \([\text{input\_length} \times 2304]\) matrix that feeds right back into the next block. The final 2304-length vector in that matrix is the most relevant to us because it 1) is the only token that sees the information from the full input sequence and 2) in the final layer it is the one that will be un-embedded and used to generate tokens. Those properties make it a good probe of the model’s internal state.
Call \(\text{state}_{i,j,k}\) the \(k\)-th decoder block’s state for \(\text{prompt}_i \times \text{style\_prompt}_j\).
\(\text{diff}_{i,j_1,k} = \text{state}_{i,j_1,k} - \operatorname{mean}_m \text{state}_{i,m,k}\) is a measure of \(\text{style\_prompt}_j\)’s effects on the model’s state at block \(k\) relative to all the other style prompts we tried.
Now, to compare how two different style prompts’ effects differ, we can calculate the cosine similarity of \(\text{diff}_{i,j_1,k}\) and \(\text{diff}_{i,j_2,k}\). Two style prompts with high cosine similarity are shifting the outputs in the same direction!
Finally, we can also compare the effects of different style prompts by comparing logit vectors immediately before sampling for the next token. Again, we can do this by subtracting the mean across all style prompts to calculate the per-style-prompt shift and then calculate cosine similarities between style prompts to compute distance.
Procedure
1) Run the model on each pair of \(\text{style\_prompt} \times \text{main\_prompt}\)
2) Record the model’s residual after each block to measure the style prompts’ effects in the internal state
3) Record each pair’s first-token logit distribution to measure the style prompts’ effects in output space
Results
Visualizing the model’s internal state
These internal states are very high dimensional, so to look at them in 2d we can run PCA. Figure 1 and 2 show each style prompt clusters’ internal state distributions over the full bank of base prompts as they progress through the model’s decoder blocks.
layer 5 none placebo plain ornate brief tokens verbose tone_formal tone_friendly cot direct careful careless layer 10 none placebo plain ornate brief tokens verbose tone_formal tone_friendly cot direct careful careless layer 15 none placebo plain ornate brief tokens verbose tone_formal tone_friendly cot direct careful careless layer 20 none placebo plain ornate brief tokens verbose tone_formal tone_friendly cot direct careful careless layer 26 (output) none placebo plain ornate brief tokens verbose tone_formal tone_friendly cot direct careful careless top-2 PCA per layer, 2σ ellipses
The first three components only explain 29% of the variation at the output layer and no more than 36% in the other layers, but even still you can see the clusters’ differences. Here is a fun interactive viewer.
Drag to rotate
Or, to get a more precise but narrower view on the same question, Figure 3 shows the cosine similarities between style prompt clusters at different block indices.
-0.5 0 0.5 0 5 10 15 20 25 plain ~ ornate, layer 0: 0 plain ~ ornate, layer 1: 0.77 plain ~ ornate, layer 2: 0.798 plain ~ ornate, layer 3: 0.918 plain ~ ornate, layer 4: 0.919 plain ~ ornate, layer 5: 0.885 plain ~ ornate, layer 6: 0.788 plain ~ ornate, layer 7: 0.746 plain ~ ornate, layer 8: 0.635 plain ~ ornate, layer 9: 0.701 plain ~ ornate, layer 10: 0.474 plain ~ ornate, layer 11: 0.404 plain ~ ornate, layer 12: 0.392 plain ~ ornate, layer 13: 0.234 plain ~ ornate, layer 14: 0.143 plain ~ ornate, layer 15: -0.11 plain ~ ornate, layer 16: -0.183 plain ~ ornate, layer 17: -0.194 plain ~ ornate, layer 18: -0.285 plain ~ ornate, layer 19: -0.277 plain ~ ornate, layer 20: -0.31 plain ~ ornate, layer 21: -0.377 plain ~ ornate, layer 22: -0.371 plain ~ ornate, layer 23: -0.368 plain ~ ornate, layer 24: -0.367 plain ~ ornate, layer 25: -0.405 plain ~ ornate, layer 26: -0.513 brief ~ verbose, layer 0: 0 brief ~ verbose, layer 1: -0.141 brief ~ verbose, layer 2: -0.031 brief ~ verbose, layer 3: 0.181 brief ~ verbose, layer 4: 0.02 brief ~ verbose, layer 5: -0.252 brief ~ verbose, layer 6: -0.089 brief ~ verbose, layer 7: -0.215 brief ~ verbose, layer 8: -0.218 brief ~ verbose, layer 9: -0.246 brief ~ verbose, layer 10: -0.613 brief ~ verbose, layer 11: -0.585 brief ~ verbose, layer 12: -0.512 brief ~ verbose, layer 13: -0.455 brief ~ verbose, layer 14: -0.418 brief ~ verbose, layer 15: -0.496 brief ~ verbose, layer 16: -0.518 brief ~ verbose, layer 17: -0.521 brief ~ verbose, layer 18: -0.584 brief ~ verbose, layer 19: -0.597 brief ~ verbose, layer 20: -0.547 brief ~ verbose, layer 21: -0.539 brief ~ verbose, layer 22: -0.578 brief ~ verbose, layer 23: -0.57 brief ~ verbose, layer 24: -0.624 brief ~ verbose, layer 25: -0.613 brief ~ verbose, layer 26: -0.536 tone_formal ~ tone_friendly, layer 0: 0 tone_formal ~ tone_friendly, layer 1: 0.598 tone_formal ~ tone_friendly, layer 2: 0.399 tone_formal ~ tone_friendly, layer 3: 0.369 tone_formal ~ tone_friendly, layer 4: 0.439 tone_formal ~ tone_friendly, layer 5: 0.498 tone_formal ~ tone_friendly, layer 6: 0.593 tone_formal ~ tone_friendly, layer 7: 0.668 tone_formal ~ tone_friendly, layer 8: 0.435 tone_formal ~ tone_friendly, layer 9: 0.463 tone_formal ~ tone_friendly, layer 10: 0.202 tone_formal ~ tone_friendly, layer 11: 0.291 tone_formal ~ tone_friendly, layer 12: 0.251 tone_formal ~ tone_friendly, layer 13: 0.103 tone_formal ~ tone_friendly, layer 14: 0.058 tone_formal ~ tone_friendly, layer 15: -0.249 tone_formal ~ tone_friendly, layer 16: -0.382 tone_formal ~ tone_friendly, layer 17: -0.441 tone_formal ~ tone_friendly, layer 18: -0.461 tone_formal ~ tone_friendly, layer 19: -0.496 tone_formal ~ tone_friendly, layer 20: -0.557 tone_formal ~ tone_friendly, layer 21: -0.593 tone_formal ~ tone_friendly, layer 22: -0.602 tone_formal ~ tone_friendly, layer 23: -0.608 tone_formal ~ tone_friendly, layer 24: -0.6 tone_formal ~ tone_friendly, layer 25: -0.613 tone_formal ~ tone_friendly, layer 26: -0.618 cot ~ direct, layer 0: 0 cot ~ direct, layer 1: 0.65 cot ~ direct, layer 2: 0.695 cot ~ direct, layer 3: 0.627 cot ~ direct, layer 4: 0.668 cot ~ direct, layer 5: 0.7 cot ~ direct, layer 6: 0.607 cot ~ direct, layer 7: 0.538 cot ~ direct, layer 8: 0.205 cot ~ direct, layer 9: 0.209 cot ~ direct, layer 10: -0.03 cot ~ direct, layer 11: 0.001 cot ~ direct, layer 12: -0.015 cot ~ direct, layer 13: -0.088 cot ~ direct, layer 14: -0.153 cot ~ direct, layer 15: -0.11 cot ~ direct, layer 16: -0.261 cot ~ direct, layer 17: -0.343 cot ~ direct, layer 18: -0.335 cot ~ direct, layer 19: -0.347 cot ~ direct, layer 20: -0.394 cot ~ direct, layer 21: -0.418 cot ~ direct, layer 22: -0.448 cot ~ direct, layer 23: -0.458 cot ~ direct, layer 24: -0.508 cot ~ direct, layer 25: -0.531 cot ~ direct, layer 26: -0.359 careful ~ careless, layer 0: 0 careful ~ careless, layer 1: -0.338 careful ~ careless, layer 2: 0.16 careful ~ careless, layer 3: 0.491 careful ~ careless, layer 4: 0.534 careful ~ careless, layer 5: 0.547 careful ~ careless, layer 6: 0.53 careful ~ careless, layer 7: 0.562 careful ~ careless, layer 8: 0.332 careful ~ careless, layer 9: 0.311 careful ~ careless, layer 10: 0.517 careful ~ careless, layer 11: 0.564 careful ~ careless, layer 12: 0.412 careful ~ careless, layer 13: 0.251 careful ~ careless, layer 14: 0.154 careful ~ careless, layer 15: 0.041 careful ~ careless, layer 16: 0.039 careful ~ careless, layer 17: 0.053 careful ~ careless, layer 18: 0.083 careful ~ careless, layer 19: 0.079 careful ~ careless, layer 20: 0.015 careful ~ careless, layer 21: 0.017 careful ~ careless, layer 22: 0.06 careful ~ careless, layer 23: 0.05 careful ~ careless, layer 24: 0.078 careful ~ careless, layer 25: 0.104 careful ~ careless, layer 26: -0.151 none ~ placebo, layer 0: 0 none ~ placebo, layer 1: -0.135 none ~ placebo, layer 2: -0.069 none ~ placebo, layer 3: -0.069 none ~ placebo, layer 4: -0.075 none ~ placebo, layer 5: -0.04 none ~ placebo, layer 6: 0.287 none ~ placebo, layer 7: 0.298 none ~ placebo, layer 8: 0.36 none ~ placebo, layer 9: 0.5 none ~ placebo, layer 10: 0.692 none ~ placebo, layer 11: 0.704 none ~ placebo, layer 12: 0.746 none ~ placebo, layer 13: 0.809 none ~ placebo, layer 14: 0.849 none ~ placebo, layer 15: 0.854 none ~ placebo, layer 16: 0.877 none ~ placebo, layer 17: 0.871 none ~ placebo, layer 18: 0.873 none ~ placebo, layer 19: 0.876 none ~ placebo, layer 20: 0.883 none ~ placebo, layer 21: 0.885 none ~ placebo, layer 22: 0.875 none ~ placebo, layer 23: 0.878 none ~ placebo, layer 24: 0.874 none ~ placebo, layer 25: 0.906 none ~ placebo, layer 26: 0.938 placebo ~ plain, layer 0: 0 placebo ~ plain, layer 1: -0.548 placebo ~ plain, layer 2: -0.416 placebo ~ plain, layer 3: -0.317 placebo ~ plain, layer 4: -0.343 placebo ~ plain, layer 5: -0.343 placebo ~ plain, layer 6: -0.461 placebo ~ plain, layer 7: -0.511 placebo ~ plain, layer 8: -0.496 placebo ~ plain, layer 9: -0.519 placebo ~ plain, layer 10: -0.638 placebo ~ plain, layer 11: -0.622 placebo ~ plain, layer 12: -0.614 placebo ~ plain, layer 13: -0.581 placebo ~ plain, layer 14: -0.537 placebo ~ plain, layer 15: -0.488 placebo ~ plain, layer 16: -0.511 placebo ~ plain, layer 17: -0.484 placebo ~ plain, layer 18: -0.471 placebo ~ plain, layer 19: -0.459 placebo ~ plain, layer 20: -0.415 placebo ~ plain, layer 21: -0.381 placebo ~ plain, layer 22: -0.377 placebo ~ plain, layer 23: -0.368 placebo ~ plain, layer 24: -0.342 placebo ~ plain, layer 25: -0.238 placebo ~ plain, layer 26: -0.108 layer cosine of cluster-mean directions band = bootstrap SE over prompts
It’s fascinating to me that opposing prompt clusters can have ~aligned activations partway through the network and then ~opposed outputs. This aligns with the understanding that early layers process the text for base meaning and then later layers plan the output. Both plain and ornate contain the phrase “mannered prose”, so the early layer alignment may be from that diction overlap.
none ~ placebo starts with mild opposition, then grows to the highest alignment on the plot. That also supports our early-layer-meaning and late-layer-output interpretation.
The logit lens
How do we know these difference vectors and their similarities mean anything useful?
Well, we can use a really cool technique called the logit lens to investifate. Essentially, we can push a style prompt’s average distance from the mean response through the same decoding-to-logits layer that text generation uses. These output logits will point at words that won’t necessarily make sense, but they will give us some indication of what the model is thinking about that layer. We passed the diff vector through the final RMSNorm before unembedding.
The table below shows the top tokens each cluster’s mean difference vector decodes to at layer 24. We chose a later layer so it’s more legible than earlier embedding layers, but we didn’t probe the output layer so we get more abstract results instead of the model’s text generation prep. The plain row is my favorite (I censored it).
Axis
Cluster
Top decoded tokens (layer 24)
mannered
plain
basic, pissed, plain, straight, guy, f**king, simple, dude, dudes, basics, Simple, basically, f**k
mannered
ornate
Dearest, dear, Ах, Lord, oh, ah, doth, esteemed, Herr, Mr, gentlemen, gentle, Oh, shall
length
brief
Brief, ито, 통해, ‘][], minimal, Box, 証拠, />);, 위해, ModelForm, 曾在, short, endforeach, katanya
length
tokens
minimal, ито, min, ミニ, eg, </blockquote>, Min, result, low, minimum, 통해, <eos>, -
length
verbose
##, #, let, Let, (#, .#, ###, ################, \#, understanding, #:
tone
tone_formal
formal, に於, esteemed, Notwithstanding, Ms, mektedir, Mr, commencing, commences, distinguished, Messrs, iż, concerning, regarding, commenced
tone
tone_friendly
hey, Hey, okay, OK, Okay, Alright, alright, ok, Heya, guys, OKAY
reasoning
cot
reasoning, Okay, ok, okay, OK, Reasoning, Ok, Alright, Here, ##, Reason, OKAY
reasoning
direct
-, •, result, />);, ?-, </blockquote>, –, −, <eos>, Box, future, ·, total
careful
careful
careful, carefully, I, While, correctly, Please, clearly, Carefully, be, accurately, Careful, To, properly, correct, As
careful
careless
ok, okay, OK, Okay, Ok, OKAY, alright, Alright, Hey
control
placebo
response, answer, request, respond, responded, responding, reply, responses, RESPOND, ##, answered, responds, requests
Do similar style prompts have similar effects? Do opposite prompts have opposite effects?
Take a given style prompt’s average effect at the final block layer across all of the base prompts in our set. Calculate the cosine similarity of those vectors against another style prompt’s average effect. Figure 4 shows that for all pairs of style prompts as a heatmap. Aligned style prompts will have positive similarity and opposing style prompts will have negative similarity.
none · none = 1 none · placebo_1 = 0.95 none · placebo_2 = 0.9 none · placebo_3 = 0.92 none · plain_1 = -0.16 none · plain_2 = -0.03 none · plain_3 = 0.29 none · ornate_1 = -0.4 none · ornate_2 = -0.42 none · ornate_3 = -0.45 none · brief_1 = 0.4 none · brief_2 = 0.28 none · brief_3 = -0.33 none · tokens_1 = -0.29 none · tokens_2 = -0.39 none · tokens_3 = -0.58 none · verbose_1 = 0.74 none · verbose_2 = 0.3 none · verbose_3 = 0.57 none · tone_formal_1 = 0.04 none · tone_formal_2 = -0.14 none · tone_formal_3 = 0.41 none · tone_friendly_1 = 0.14 none · tone_friendly_2 = 0.1 none · tone_friendly_3 = 0.08 none · cot_1 = 0.08 none · cot_2 = 0.22 none · cot_3 = 0.08 none · direct_1 = -0.06 none · direct_2 = -0.56 none · direct_3 = -0.29 none · careful_1 = 0.77 none · careful_2 = 0.9 none · careful_3 = 0.75 none · careless_1 = -0.38 none · careless_2 = 0.04 none · careless_3 = 0.66 placebo_1 · none = 0.95 placebo_1 · placebo_1 = 1 placebo_1 · placebo_2 = 0.97 placebo_1 · placebo_3 = 0.97 placebo_1 · plain_1 = -0.23 placebo_1 · plain_2 = -0.1 placebo_1 · plain_3 = 0.21 placebo_1 · ornate_1 = -0.4 placebo_1 · ornate_2 = -0.4 placebo_1 · ornate_3 = -0.47 placebo_1 · brief_1 = 0.3 placebo_1 · brief_2 = 0.18 placebo_1 · brief_3 = -0.43 placebo_1 · tokens_1 = -0.33 placebo_1 · tokens_2 = -0.44 placebo_1 · tokens_3 = -0.59 placebo_1 · verbose_1 = 0.76 placebo_1 · verbose_2 = 0.39 placebo_1 · verbose_3 = 0.63 placebo_1 · tone_formal_1 = 0.04 placebo_1 · tone_formal_2 = -0.1 placebo_1 · tone_formal_3 = 0.4 placebo_1 · tone_friendly_1 = 0.13 placebo_1 · tone_friendly_2 = 0.06 placebo_1 · tone_friendly_3 = 0.04 placebo_1 · cot_1 = 0.17 placebo_1 · cot_2 = 0.37 placebo_1 · cot_3 = 0.16 placebo_1 · direct_1 = -0.11 placebo_1 · direct_2 = -0.55 placebo_1 · direct_3 = -0.35 placebo_1 · careful_1 = 0.78 placebo_1 · careful_2 = 0.9 placebo_1 · careful_3 = 0.78 placebo_1 · careless_1 = -0.36 placebo_1 · careless_2 = 0.05 placebo_1 · careless_3 = 0.62 placebo_2 · none = 0.9 placebo_2 · placebo_1 = 0.97 placebo_2 · placebo_2 = 1 placebo_2 · placebo_3 = 0.93 placebo_2 · plain_1 = -0.24 placebo_2 · plain_2 = -0.16 placebo_2 · plain_3 = 0.15 placebo_2 · ornate_1 = -0.4 placebo_2 · ornate_2 = -0.38 placebo_2 · ornate_3 = -0.46 placebo_2 · brief_1 = 0.25 placebo_2 · brief_2 = 0.1 placebo_2 · brief_3 = -0.48 placebo_2 · tokens_1 = -0.35 placebo_2 · tokens_2 = -0.44 placebo_2 · tokens_3 = -0.59 placebo_2 · verbose_1 = 0.74 placebo_2 · verbose_2 = 0.4 placebo_2 · verbose_3 = 0.64 placebo_2 · tone_formal_1 = -0.01 placebo_2 · tone_formal_2 = -0.13 placebo_2 · tone_formal_3 = 0.35 placebo_2 · tone_friendly_1 = 0.21 placebo_2 · tone_friendly_2 = 0.12 placebo_2 · tone_friendly_3 = 0.11 placebo_2 · cot_1 = 0.19 placebo_2 · cot_2 = 0.4 placebo_2 · cot_3 = 0.16 placebo_2 · direct_1 = -0.19 placebo_2 · direct_2 = -0.59 placebo_2 · direct_3 = -0.4 placebo_2 · careful_1 = 0.73 placebo_2 · careful_2 = 0.86 placebo_2 · careful_3 = 0.76 placebo_2 · careless_1 = -0.31 placebo_2 · careless_2 = 0.07 placebo_2 · careless_3 = 0.63 placebo_3 · none = 0.92 placebo_3 · placebo_1 = 0.97 placebo_3 · placebo_2 = 0.93 placebo_3 · placebo_3 = 1 placebo_3 · plain_1 = -0.23 placebo_3 · plain_2 = -0.09 placebo_3 · plain_3 = 0.21 placebo_3 · ornate_1 = -0.38 placebo_3 · ornate_2 = -0.4 placebo_3 · ornate_3 = -0.46 placebo_3 · brief_1 = 0.26 placebo_3 · brief_2 = 0.16 placebo_3 · brief_3 = -0.43 placebo_3 · tokens_1 = -0.32 placebo_3 · tokens_2 = -0.45 placebo_3 · tokens_3 = -0.57 placebo_3 · verbose_1 = 0.79 placebo_3 · verbose_2 = 0.42 placebo_3 · verbose_3 = 0.65 placebo_3 · tone_formal_1 = 0.09 placebo_3 · tone_formal_2 = -0.04 placebo_3 · tone_formal_3 = 0.43 placebo_3 · tone_friendly_1 = 0.05 placebo_3 · tone_friendly_2 = -0.01 placebo_3 · tone_friendly_3 = -0.03 placebo_3 · cot_1 = 0.17 placebo_3 · cot_2 = 0.38 placebo_3 · cot_3 = 0.18 placebo_3 · direct_1 = -0.08 placebo_3 · direct_2 = -0.51 placebo_3 · direct_3 = -0.34 placebo_3 · careful_1 = 0.82 placebo_3 · careful_2 = 0.86 placebo_3 · careful_3 = 0.79 placebo_3 · careless_1 = -0.33 placebo_3 · careless_2 = 0.05 placebo_3 · careless_3 = 0.56 plain_1 · none = -0.16 plain_1 · placebo_1 = -0.23 plain_1 · placebo_2 = -0.24 plain_1 · placebo_3 = -0.23 plain_1 · plain_1 = 1 plain_1 · plain_2 = 0.69 plain_1 · plain_3 = 0.45 plain_1 · ornate_1 = -0.53 plain_1 · ornate_2 = -0.44 plain_1 · ornate_3 = -0.27 plain_1 · brief_1 = 0.38 plain_1 · brief_2 = 0.37 plain_1 · brief_3 = 0.49 plain_1 · tokens_1 = 0.5 plain_1 · tokens_2 = 0.55 plain_1 · tokens_3 = 0.43 plain_1 · verbose_1 = -0.33 plain_1 · verbose_2 = -0.37 plain_1 · verbose_3 = -0.43 plain_1 · tone_formal_1 = -0.5 plain_1 · tone_formal_2 = -0.49 plain_1 · tone_formal_3 = -0.43 plain_1 · tone_friendly_1 = 0.02 plain_1 · tone_friendly_2 = 0.23 plain_1 · tone_friendly_3 = 0.18 plain_1 · cot_1 = -0.15 plain_1 · cot_2 = -0.3 plain_1 · cot_3 = 0 plain_1 · direct_1 = 0.2 plain_1 · direct_2 = 0.29 plain_1 · direct_3 = 0.47 plain_1 · careful_1 = -0.25 plain_1 · careful_2 = -0.36 plain_1 · careful_3 = -0.27 plain_1 · careless_1 = 0.22 plain_1 · careless_2 = 0.52 plain_1 · careless_3 = 0.16 plain_2 · none = -0.03 plain_2 · placebo_1 = -0.1 plain_2 · placebo_2 = -0.16 plain_2 · placebo_3 = -0.09 plain_2 · plain_1 = 0.69 plain_2 · plain_2 = 1 plain_2 · plain_3 = 0.81 plain_2 · ornate_1 = -0.32 plain_2 · ornate_2 = -0.52 plain_2 · ornate_3 = -0.38 plain_2 · brief_1 = 0.48 plain_2 · brief_2 = 0.51 plain_2 · brief_3 = 0.56 plain_2 · tokens_1 = 0.41 plain_2 · tokens_2 = 0.39 plain_2 · tokens_3 = 0.3 plain_2 · verbose_1 = -0.3 plain_2 · verbose_2 = -0.41 plain_2 · verbose_3 = -0.4 plain_2 · tone_formal_1 = -0.1 plain_2 · tone_formal_2 = -0.2 plain_2 · tone_formal_3 = 0.07 plain_2 · tone_friendly_1 = -0.19 plain_2 · tone_friendly_2 = 0.01 plain_2 · tone_friendly_3 = -0.04 plain_2 · cot_1 = -0.12 plain_2 · cot_2 = -0.32 plain_2 · cot_3 = -0.03 plain_2 · direct_1 = 0.45 plain_2 · direct_2 = 0.36 plain_2 · direct_3 = 0.55 plain_2 · careful_1 = 0.01 plain_2 · careful_2 = -0.06 plain_2 · careful_3 = -0.1 plain_2 · careless_1 = -0.11 plain_2 · careless_2 = 0.1 plain_2 · careless_3 = -0.02 plain_3 · none = 0.29 plain_3 · placebo_1 = 0.21 plain_3 · placebo_2 = 0.15 plain_3 · placebo_3 = 0.21 plain_3 · plain_1 = 0.45 plain_3 · plain_2 = 0.81 plain_3 · plain_3 = 1 plain_3 · ornate_1 = -0.29 plain_3 · ornate_2 = -0.56 plain_3 · ornate_3 = -0.41 plain_3 · brief_1 = 0.59 plain_3 · brief_2 = 0.58 plain_3 · brief_3 = 0.38 plain_3 · tokens_1 = 0.2 plain_3 · tokens_2 = 0.15 plain_3 · tokens_3 = -0.01 plain_3 · verbose_1 = -0.03 plain_3 · verbose_2 = -0.26 plain_3 · verbose_3 = -0.16 plain_3 · tone_formal_1 = 0.05 plain_3 · tone_formal_2 = -0.14 plain_3 · tone_formal_3 = 0.34 plain_3 · tone_friendly_1 = -0.16 plain_3 · tone_friendly_2 = 0.02 plain_3 · tone_friendly_3 = -0.03 plain_3 · cot_1 = -0.07 plain_3 · cot_2 = -0.21 plain_3 · cot_3 = -0.03 plain_3 · direct_1 = 0.33 plain_3 · direct_2 = 0.1 plain_3 · direct_3 = 0.3 plain_3 · careful_1 = 0.28 plain_3 · careful_2 = 0.3 plain_3 · careful_3 = 0.17 plain_3 · careless_1 = -0.36 plain_3 · careless_2 = 0.01 plain_3 · careless_3 = 0.16 ornate_1 · none = -0.4 ornate_1 · placebo_1 = -0.4 ornate_1 · placebo_2 = -0.4 ornate_1 · placebo_3 = -0.38 ornate_1 · plain_1 = -0.53 ornate_1 · plain_2 = -0.32 ornate_1 · plain_3 = -0.29 ornate_1 · ornate_1 = 1 ornate_1 · ornate_2 = 0.83 ornate_1 · ornate_3 = 0.74 ornate_1 · brief_1 = -0.44 ornate_1 · brief_2 = -0.33 ornate_1 · brief_3 = -0.09 ornate_1 · tokens_1 = -0.38 ornate_1 · tokens_2 = -0.35 ornate_1 · tokens_3 = -0.17 ornate_1 · verbose_1 = -0.16 ornate_1 · verbose_2 = -0.05 ornate_1 · verbose_3 = -0.08 ornate_1 · tone_formal_1 = 0.59 ornate_1 · tone_formal_2 = 0.64 ornate_1 · tone_formal_3 = 0.26 ornate_1 · tone_friendly_1 = -0.3 ornate_1 · tone_friendly_2 = -0.32 ornate_1 · tone_friendly_3 = -0.27 ornate_1 · cot_1 = -0.28 ornate_1 · cot_2 = -0.27 ornate_1 · cot_3 = -0.37 ornate_1 · direct_1 = 0.02 ornate_1 · direct_2 = 0.04 ornate_1 · direct_3 = -0.14 ornate_1 · careful_1 = -0.2 ornate_1 · careful_2 = -0.18 ornate_1 · careful_3 = -0.27 ornate_1 · careless_1 = -0.17 ornate_1 · careless_2 = -0.6 ornate_1 · careless_3 = -0.51 ornate_2 · none = -0.42 ornate_2 · placebo_1 = -0.4 ornate_2 · placebo_2 = -0.38 ornate_2 · placebo_3 = -0.4 ornate_2 · plain_1 = -0.44 ornate_2 · plain_2 = -0.52 ornate_2 · plain_3 = -0.56 ornate_2 · ornate_1 = 0.83 ornate_2 · ornate_2 = 1 ornate_2 · ornate_3 = 0.8 ornate_2 · brief_1 = -0.53 ornate_2 · brief_2 = -0.48 ornate_2 · brief_3 = -0.21 ornate_2 · tokens_1 = -0.41 ornate_2 · tokens_2 = -0.32 ornate_2 · tokens_3 = -0.17 ornate_2 · verbose_1 = -0.15 ornate_2 · verbose_2 = -0.02 ornate_2 · verbose_3 = -0.08 ornate_2 · tone_formal_1 = 0.3 ornate_2 · tone_formal_2 = 0.41 ornate_2 · tone_formal_3 = -0.08 ornate_2 · tone_friendly_1 = -0.12 ornate_2 · tone_friendly_2 = -0.17 ornate_2 · tone_friendly_3 = -0.13 ornate_2 · cot_1 = -0.27 ornate_2 · cot_2 = -0.2 ornate_2 · cot_3 = -0.32 ornate_2 · direct_1 = -0.15 ornate_2 · direct_2 = -0.07 ornate_2 · direct_3 = -0.22 ornate_2 · careful_1 = -0.32 ornate_2 · careful_2 = -0.31 ornate_2 · careful_3 = -0.34 ornate_2 · careless_1 = 0.03 ornate_2 · careless_2 = -0.39 ornate_2 · careless_3 = -0.37 ornate_3 · none = -0.45 ornate_3 · placebo_1 = -0.47 ornate_3 · placebo_2 = -0.46 ornate_3 · placebo_3 = -0.46 ornate_3 · plain_1 = -0.27 ornate_3 · plain_2 = -0.38 ornate_3 · plain_3 = -0.41 ornate_3 · ornate_1 = 0.74 ornate_3 · ornate_2 = 0.8 ornate_3 · ornate_3 = 1 ornate_3 · brief_1 = -0.37 ornate_3 · brief_2 = -0.31 ornate_3 · brief_3 = 0.01 ornate_3 · tokens_1 = -0.27 ornate_3 · tokens_2 = -0.17 ornate_3 · tokens_3 = -0.04 ornate_3 · verbose_1 = -0.25 ornate_3 · verbose_2 = -0.13 ornate_3 · verbose_3 = -0.16 ornate_3 · tone_formal_1 = 0.19 ornate_3 · tone_formal_2 = 0.25 ornate_3 · tone_formal_3 = -0.18 ornate_3 · tone_friendly_1 = -0.17 ornate_3 · tone_friendly_2 = -0.13 ornate_3 · tone_friendly_3 = -0.09 ornate_3 · cot_1 = -0.41 ornate_3 · cot_2 = -0.38 ornate_3 · cot_3 = -0.44 ornate_3 · direct_1 = 0 ornate_3 · direct_2 = 0.07 ornate_3 · direct_3 = -0.07 ornate_3 · careful_1 = -0.35 ornate_3 · careful_2 = -0.32 ornate_3 · careful_3 = -0.41 ornate_3 · careless_1 = 0.03 ornate_3 · careless_2 = -0.37 ornate_3 · careless_3 = -0.3 brief_1 · none = 0.4 brief_1 · placebo_1 = 0.3 brief_1 · placebo_2 = 0.25 brief_1 · placebo_3 = 0.26 brief_1 · plain_1 = 0.38 brief_1 · plain_2 = 0.48 brief_1 · plain_3 = 0.59 brief_1 · ornate_1 = -0.44 brief_1 · ornate_2 = -0.53 brief_1 · ornate_3 = -0.37 brief_1 · brief_1 = 1 brief_1 · brief_2 = 0.92 brief_1 · brief_3 = 0.56 brief_1 · tokens_1 = 0.48 brief_1 · tokens_2 = 0.44 brief_1 · tokens_3 = 0.16 brief_1 · verbose_1 = -0.08 brief_1 · verbose_2 = -0.44 brief_1 · verbose_3 = -0.28 brief_1 · tone_formal_1 = -0.18 brief_1 · tone_formal_2 = -0.34 brief_1 · tone_formal_3 = 0.14 brief_1 · tone_friendly_1 = 0 brief_1 · tone_friendly_2 = 0.08 brief_1 · tone_friendly_3 = 0.01 brief_1 · cot_1 = -0.21 brief_1 · cot_2 = -0.35 brief_1 · cot_3 = -0.19 brief_1 · direct_1 = 0.44 brief_1 · direct_2 = 0.09 brief_1 · direct_3 = 0.46 brief_1 · careful_1 = 0.23 brief_1 · careful_2 = 0.28 brief_1 · careful_3 = 0.14 brief_1 · careless_1 = -0.38 brief_1 · careless_2 = 0.19 brief_1 · careless_3 = 0.35 brief_2 · none = 0.28 brief_2 · placebo_1 = 0.18 brief_2 · placebo_2 = 0.1 brief_2 · placebo_3 = 0.16 brief_2 · plain_1 = 0.37 brief_2 · plain_2 = 0.51 brief_2 · plain_3 = 0.58 brief_2 · ornate_1 = -0.33 brief_2 · ornate_2 = -0.48 brief_2 · ornate_3 = -0.31 brief_2 · brief_1 = 0.92 brief_2 · brief_2 = 1 brief_2 · brief_3 = 0.7 brief_2 · tokens_1 = 0.56 brief_2 · tokens_2 = 0.48 brief_2 · tokens_3 = 0.28 brief_2 · verbose_1 = -0.1 brief_2 · verbose_2 = -0.44 brief_2 · verbose_3 = -0.31 brief_2 · tone_formal_1 = 0.03 brief_2 · tone_formal_2 = -0.13 brief_2 · tone_formal_3 = 0.25 brief_2 · tone_friendly_1 = -0.27 brief_2 · tone_friendly_2 = -0.18 brief_2 · tone_friendly_3 = -0.26 brief_2 · cot_1 = -0.23 brief_2 · cot_2 = -0.38 brief_2 · cot_3 = -0.2 brief_2 · direct_1 = 0.58 brief_2 · direct_2 = 0.28 brief_2 · direct_3 = 0.59 brief_2 · careful_1 = 0.24 brief_2 · careful_2 = 0.17 brief_2 · careful_3 = 0.12 brief_2 · careless_1 = -0.4 brief_2 · careless_2 = 0.15 brief_2 · careless_3 = 0.09 brief_3 · none = -0.33 brief_3 · placebo_1 = -0.43 brief_3 · placebo_2 = -0.48 brief_3 · placebo_3 = -0.43 brief_3 · plain_1 = 0.49 brief_3 · plain_2 = 0.56 brief_3 · plain_3 = 0.38 brief_3 · ornate_1 = -0.09 brief_3 · ornate_2 = -0.21 brief_3 · ornate_3 = 0.01 brief_3 · brief_1 = 0.56 brief_3 · brief_2 = 0.7 brief_3 · brief_3 = 1 brief_3 · tokens_1 = 0.68 brief_3 · tokens_2 = 0.69 brief_3 · tokens_3 = 0.66 brief_3 · verbose_1 = -0.59 brief_3 · verbose_2 = -0.65 brief_3 · verbose_3 = -0.67 brief_3 · tone_formal_1 = -0.07 brief_3 · tone_formal_2 = -0.15 brief_3 · tone_formal_3 = -0.09 brief_3 · tone_friendly_1 = -0.31 brief_3 · tone_friendly_2 = -0.18 brief_3 · tone_friendly_3 = -0.24 brief_3 · cot_1 = -0.33 brief_3 · cot_2 = -0.57 brief_3 · cot_3 = -0.3 brief_3 · direct_1 = 0.71 brief_3 · direct_2 = 0.68 brief_3 · direct_3 = 0.87 brief_3 · careful_1 = -0.27 brief_3 · careful_2 = -0.37 brief_3 · careful_3 = -0.4 brief_3 · careless_1 = -0.18 brief_3 · careless_2 = 0.07 brief_3 · careless_3 = -0.27 tokens_1 · none = -0.29 tokens_1 · placebo_1 = -0.33 tokens_1 · placebo_2 = -0.35 tokens_1 · placebo_3 = -0.32 tokens_1 · plain_1 = 0.5 tokens_1 · plain_2 = 0.41 tokens_1 · plain_3 = 0.2 tokens_1 · ornate_1 = -0.38 tokens_1 · ornate_2 = -0.41 tokens_1 · ornate_3 = -0.27 tokens_1 · brief_1 = 0.48 tokens_1 · brief_2 = 0.56 tokens_1 · brief_3 = 0.68 tokens_1 · tokens_1 = 1 tokens_1 · tokens_2 = 0.95 tokens_1 · tokens_3 = 0.87 tokens_1 · verbose_1 = -0.51 tokens_1 · verbose_2 = -0.46 tokens_1 · verbose_3 = -0.53 tokens_1 · tone_formal_1 = -0.31 tokens_1 · tone_formal_2 = -0.3 tokens_1 · tone_formal_3 = -0.25 tokens_1 · tone_friendly_1 = -0.14 tokens_1 · tone_friendly_2 = -0.11 tokens_1 · tone_friendly_3 = -0.15 tokens_1 · cot_1 = -0.1 tokens_1 · cot_2 = -0.24 tokens_1 · cot_3 = -0.06 tokens_1 · direct_1 = 0.33 tokens_1 · direct_2 = 0.53 tokens_1 · direct_3 = 0.6 tokens_1 · careful_1 = -0.3 tokens_1 · careful_2 = -0.42 tokens_1 · careful_3 = -0.32 tokens_1 · careless_1 = 0.17 tokens_1 · careless_2 = 0.36 tokens_1 · careless_3 = -0.2 tokens_2 · none = -0.39 tokens_2 · placebo_1 = -0.44 tokens_2 · placebo_2 = -0.44 tokens_2 · placebo_3 = -0.45 tokens_2 · plain_1 = 0.55 tokens_2 · plain_2 = 0.39 tokens_2 · plain_3 = 0.15 tokens_2 · ornate_1 = -0.35 tokens_2 · ornate_2 = -0.32 tokens_2 · ornate_3 = -0.17 tokens_2 · brief_1 = 0.44 tokens_2 · brief_2 = 0.48 tokens_2 · brief_3 = 0.69 tokens_2 · tokens_1 = 0.95 tokens_2 · tokens_2 = 1 tokens_2 · tokens_3 = 0.87 tokens_2 · verbose_1 = -0.6 tokens_2 · verbose_2 = -0.51 tokens_2 · verbose_3 = -0.61 tokens_2 · tone_formal_1 = -0.42 tokens_2 · tone_formal_2 = -0.37 tokens_2 · tone_formal_3 = -0.41 tokens_2 · tone_friendly_1 = -0.03 tokens_2 · tone_friendly_2 = 0.01 tokens_2 · tone_friendly_3 = -0.04 tokens_2 · cot_1 = -0.16 tokens_2 · cot_2 = -0.31 tokens_2 · cot_3 = -0.12 tokens_2 · direct_1 = 0.27 tokens_2 · direct_2 = 0.51 tokens_2 · direct_3 = 0.59 tokens_2 · careful_1 = -0.44 tokens_2 · careful_2 = -0.52 tokens_2 · careful_3 = -0.43 tokens_2 · careless_1 = 0.26 tokens_2 · careless_2 = 0.37 tokens_2 · careless_3 = -0.15 tokens_3 · none = -0.58 tokens_3 · placebo_1 = -0.59 tokens_3 · placebo_2 = -0.59 tokens_3 · placebo_3 = -0.57 tokens_3 · plain_1 = 0.43 tokens_3 · plain_2 = 0.3 tokens_3 · plain_3 = -0.01 tokens_3 · ornate_1 = -0.17 tokens_3 · ornate_2 = -0.17 tokens_3 · ornate_3 = -0.04 tokens_3 · brief_1 = 0.16 tokens_3 · brief_2 = 0.28 tokens_3 · brief_3 = 0.66 tokens_3 · tokens_1 = 0.87 tokens_3 · tokens_2 = 0.87 tokens_3 · tokens_3 = 1 tokens_3 · verbose_1 = -0.65 tokens_3 · verbose_2 = -0.43 tokens_3 · verbose_3 = -0.58 tokens_3 · tone_formal_1 = -0.3 tokens_3 · tone_formal_2 = -0.22 tokens_3 · tone_formal_3 = -0.4 tokens_3 · tone_friendly_1 = -0.18 tokens_3 · tone_friendly_2 = -0.14 tokens_3 · tone_friendly_3 = -0.16 tokens_3 · cot_1 = -0.14 tokens_3 · cot_2 = -0.25 tokens_3 · cot_3 = -0.1 tokens_3 · direct_1 = 0.28 tokens_3 · direct_2 = 0.67 tokens_3 · direct_3 = 0.58 tokens_3 · careful_1 = -0.5 tokens_3 · careful_2 = -0.65 tokens_3 · careful_3 = -0.5 tokens_3 · careless_1 = 0.35 tokens_3 · careless_2 = 0.25 tokens_3 · careless_3 = -0.38 verbose_1 · none = 0.74 verbose_1 · placebo_1 = 0.76 verbose_1 · placebo_2 = 0.74 verbose_1 · placebo_3 = 0.79 verbose_1 · plain_1 = -0.33 verbose_1 · plain_2 = -0.3 verbose_1 · plain_3 = -0.03 verbose_1 · ornate_1 = -0.16 verbose_1 · ornate_2 = -0.15 verbose_1 · ornate_3 = -0.25 verbose_1 · brief_1 = -0.08 verbose_1 · brief_2 = -0.1 verbose_1 · brief_3 = -0.59 verbose_1 · tokens_1 = -0.51 verbose_1 · tokens_2 = -0.6 verbose_1 · tokens_3 = -0.65 verbose_1 · verbose_1 = 1 verbose_1 · verbose_2 = 0.76 verbose_1 · verbose_3 = 0.91 verbose_1 · tone_formal_1 = 0.28 verbose_1 · tone_formal_2 = 0.21 verbose_1 · tone_formal_3 = 0.44 verbose_1 · tone_friendly_1 = -0.07 verbose_1 · tone_friendly_2 = -0.16 verbose_1 · tone_friendly_3 = -0.14 verbose_1 · cot_1 = 0.19 verbose_1 · cot_2 = 0.44 verbose_1 · cot_3 = 0.19 verbose_1 · direct_1 = -0.28 verbose_1 · direct_2 = -0.56 verbose_1 · direct_3 = -0.52 verbose_1 · careful_1 = 0.73 verbose_1 · careful_2 = 0.7 verbose_1 · careful_3 = 0.74 verbose_1 · careless_1 = -0.26 verbose_1 · careless_2 = 0.04 verbose_1 · careless_3 = 0.35 verbose_2 · none = 0.3 verbose_2 · placebo_1 = 0.39 verbose_2 · placebo_2 = 0.4 verbose_2 · placebo_3 = 0.42 verbose_2 · plain_1 = -0.37 verbose_2 · plain_2 = -0.41 verbose_2 · plain_3 = -0.26 verbose_2 · ornate_1 = -0.05 verbose_2 · ornate_2 = -0.02 verbose_2 · ornate_3 = -0.13 verbose_2 · brief_1 = -0.44 verbose_2 · brief_2 = -0.44 verbose_2 · brief_3 = -0.65 verbose_2 · tokens_1 = -0.46 verbose_2 · tokens_2 = -0.51 verbose_2 · tokens_3 = -0.43 verbose_2 · verbose_1 = 0.76 verbose_2 · verbose_2 = 1 verbose_2 · verbose_3 = 0.89 verbose_2 · tone_formal_1 = 0.15 verbose_2 · tone_formal_2 = 0.2 verbose_2 · tone_formal_3 = 0.16 verbose_2 · tone_friendly_1 = -0.07 verbose_2 · tone_friendly_2 = -0.18 verbose_2 · tone_friendly_3 = -0.12 verbose_2 · cot_1 = 0.36 verbose_2 · cot_2 = 0.64 verbose_2 · cot_3 = 0.28 verbose_2 · direct_1 = -0.46 verbose_2 · direct_2 = -0.36 verbose_2 · direct_3 = -0.59 verbose_2 · careful_1 = 0.39 verbose_2 · careful_2 = 0.32 verbose_2 · careful_3 = 0.47 verbose_2 · careless_1 = 0.05 verbose_2 · careless_2 = 0.06 verbose_2 · careless_3 = 0.09 verbose_3 · none = 0.57 verbose_3 · placebo_1 = 0.63 verbose_3 · placebo_2 = 0.64 verbose_3 · placebo_3 = 0.65 verbose_3 · plain_1 = -0.43 verbose_3 · plain_2 = -0.4 verbose_3 · plain_3 = -0.16 verbose_3 · ornate_1 = -0.08 verbose_3 · ornate_2 = -0.08 verbose_3 · ornate_3 = -0.16 verbose_3 · brief_1 = -0.28 verbose_3 · brief_2 = -0.31 verbose_3 · brief_3 = -0.67 verbose_3 · tokens_1 = -0.53 verbose_3 · tokens_2 = -0.61 verbose_3 · tokens_3 = -0.58 verbose_3 · verbose_1 = 0.91 verbose_3 · verbose_2 = 0.89 verbose_3 · verbose_3 = 1 verbose_3 · tone_formal_1 = 0.22 verbose_3 · tone_formal_2 = 0.19 verbose_3 · tone_formal_3 = 0.31 verbose_3 · tone_friendly_1 = -0.06 verbose_3 · tone_friendly_2 = -0.16 verbose_3 · tone_friendly_3 = -0.1 verbose_3 · cot_1 = 0.27 verbose_3 · cot_2 = 0.57 verbose_3 · cot_3 = 0.22 verbose_3 · direct_1 = -0.41 verbose_3 · direct_2 = -0.5 verbose_3 · direct_3 = -0.62 verbose_3 · careful_1 = 0.59 verbose_3 · careful_2 = 0.61 verbose_3 · careful_3 = 0.67 verbose_3 · careless_1 = -0.1 verbose_3 · careless_2 = -0.01 verbose_3 · careless_3 = 0.24 tone_formal_1 · none = 0.04 tone_formal_1 · placebo_1 = 0.04 tone_formal_1 · placebo_2 = -0.01 tone_formal_1 · placebo_3 = 0.09 tone_formal_1 · plain_1 = -0.5 tone_formal_1 · plain_2 = -0.1 tone_formal_1 · plain_3 = 0.05 tone_formal_1 · ornate_1 = 0.59 tone_formal_1 · ornate_2 = 0.3 tone_formal_1 · ornate_3 = 0.19 tone_formal_1 · brief_1 = -0.18 tone_formal_1 · brief_2 = 0.03 tone_formal_1 · brief_3 = -0.07 tone_formal_1 · tokens_1 = -0.31 tone_formal_1 · tokens_2 = -0.42 tone_formal_1 · tokens_3 = -0.3 tone_formal_1 · verbose_1 = 0.28 tone_formal_1 · verbose_2 = 0.15 tone_formal_1 · verbose_3 = 0.22 tone_formal_1 · tone_formal_1 = 1 tone_formal_1 · tone_formal_2 = 0.94 tone_formal_1 · tone_formal_3 = 0.82 tone_formal_1 · tone_friendly_1 = -0.58 tone_formal_1 · tone_friendly_2 = -0.63 tone_formal_1 · tone_friendly_3 = -0.63 tone_formal_1 · cot_1 = -0.06 tone_formal_1 · cot_2 = -0.06 tone_formal_1 · cot_3 = -0.12 tone_formal_1 · direct_1 = 0.19 tone_formal_1 · direct_2 = 0.03 tone_formal_1 · direct_3 = -0.03 tone_formal_1 · careful_1 = 0.35 tone_formal_1 · careful_2 = 0.2 tone_formal_1 · careful_3 = 0.23 tone_formal_1 · careless_1 = -0.43 tone_formal_1 · careless_2 = -0.45 tone_formal_1 · careless_3 = -0.5 tone_formal_2 · none = -0.14 tone_formal_2 · placebo_1 = -0.1 tone_formal_2 · placebo_2 = -0.13 tone_formal_2 · placebo_3 = -0.04 tone_formal_2 · plain_1 = -0.49 tone_formal_2 · plain_2 = -0.2 tone_formal_2 · plain_3 = -0.14 tone_formal_2 · ornate_1 = 0.64 tone_formal_2 · ornate_2 = 0.41 tone_formal_2 · ornate_3 = 0.25 tone_formal_2 · brief_1 = -0.34 tone_formal_2 · brief_2 = -0.13 tone_formal_2 · brief_3 = -0.15 tone_formal_2 · tokens_1 = -0.3 tone_formal_2 · tokens_2 = -0.37 tone_formal_2 · tokens_3 = -0.22 tone_formal_2 · verbose_1 = 0.21 tone_formal_2 · verbose_2 = 0.2 tone_formal_2 · verbose_3 = 0.19 tone_formal_2 · tone_formal_1 = 0.94 tone_formal_2 · tone_formal_2 = 1 tone_formal_2 · tone_formal_3 = 0.66 tone_formal_2 · tone_friendly_1 = -0.57 tone_formal_2 · tone_friendly_2 = -0.65 tone_formal_2 · tone_friendly_3 = -0.63 tone_formal_2 · cot_1 = -0.05 tone_formal_2 · cot_2 = 0 tone_formal_2 · cot_3 = -0.08 tone_formal_2 · direct_1 = 0.09 tone_formal_2 · direct_2 = 0.09 tone_formal_2 · direct_3 = -0.08 tone_formal_2 · careful_1 = 0.21 tone_formal_2 · careful_2 = -0.01 tone_formal_2 · careful_3 = 0.11 tone_formal_2 · careless_1 = -0.23 tone_formal_2 · careless_2 = -0.38 tone_formal_2 · careless_3 = -0.59 tone_formal_3 · none = 0.41 tone_formal_3 · placebo_1 = 0.4 tone_formal_3 · placebo_2 = 0.35 tone_formal_3 · placebo_3 = 0.43 tone_formal_3 · plain_1 = -0.43 tone_formal_3 · plain_2 = 0.07 tone_formal_3 · plain_3 = 0.34 tone_formal_3 · ornate_1 = 0.26 tone_formal_3 · ornate_2 = -0.08 tone_formal_3 · ornate_3 = -0.18 tone_formal_3 · brief_1 = 0.14 tone_formal_3 · brief_2 = 0.25 tone_formal_3 · brief_3 = -0.09 tone_formal_3 · tokens_1 = -0.25 tone_formal_3 · tokens_2 = -0.41 tone_formal_3 · tokens_3 = -0.4 tone_formal_3 · verbose_1 = 0.44 tone_formal_3 · verbose_2 = 0.16 tone_formal_3 · verbose_3 = 0.31 tone_formal_3 · tone_formal_1 = 0.82 tone_formal_3 · tone_formal_2 = 0.66 tone_formal_3 · tone_formal_3 = 1 tone_formal_3 · tone_friendly_1 = -0.38 tone_formal_3 · tone_friendly_2 = -0.43 tone_formal_3 · tone_friendly_3 = -0.44 tone_formal_3 · cot_1 = 0.03 tone_formal_3 · cot_2 = 0.03 tone_formal_3 · cot_3 = -0.03 tone_formal_3 · direct_1 = 0.19 tone_formal_3 · direct_2 = -0.14 tone_formal_3 · direct_3 = -0.07 tone_formal_3 · careful_1 = 0.59 tone_formal_3 · careful_2 = 0.54 tone_formal_3 · careful_3 = 0.46 tone_formal_3 · careless_1 = -0.6 tone_formal_3 · careless_2 = -0.37 tone_formal_3 · careless_3 = -0.17 tone_friendly_1 · none = 0.14 tone_friendly_1 · placebo_1 = 0.13 tone_friendly_1 · placebo_2 = 0.21 tone_friendly_1 · placebo_3 = 0.05 tone_friendly_1 · plain_1 = 0.02 tone_friendly_1 · plain_2 = -0.19 tone_friendly_1 · plain_3 = -0.16 tone_friendly_1 · ornate_1 = -0.3 tone_friendly_1 · ornate_2 = -0.12 tone_friendly_1 · ornate_3 = -0.17 tone_friendly_1 · brief_1 = 0 tone_friendly_1 · brief_2 = -0.27 tone_friendly_1 · brief_3 = -0.31 tone_friendly_1 · tokens_1 = -0.14 tone_friendly_1 · tokens_2 = -0.03 tone_friendly_1 · tokens_3 = -0.18 tone_friendly_1 · verbose_1 = -0.07 tone_friendly_1 · verbose_2 = -0.07 tone_friendly_1 · verbose_3 = -0.06 tone_friendly_1 · tone_formal_1 = -0.58 tone_friendly_1 · tone_formal_2 = -0.57 tone_friendly_1 · tone_formal_3 = -0.38 tone_friendly_1 · tone_friendly_1 = 1 tone_friendly_1 · tone_friendly_2 = 0.91 tone_friendly_1 · tone_friendly_3 = 0.88 tone_friendly_1 · cot_1 = 0.04 tone_friendly_1 · cot_2 = 0.05 tone_friendly_1 · cot_3 = -0.01 tone_friendly_1 · direct_1 = -0.45 tone_friendly_1 · direct_2 = -0.49 tone_friendly_1 · direct_3 = -0.36 tone_friendly_1 · careful_1 = -0.27 tone_friendly_1 · careful_2 = 0.07 tone_friendly_1 · careful_3 = -0.14 tone_friendly_1 · careless_1 = 0.13 tone_friendly_1 · careless_2 = 0.05 tone_friendly_1 · careless_3 = 0.54 tone_friendly_2 · none = 0.1 tone_friendly_2 · placebo_1 = 0.06 tone_friendly_2 · placebo_2 = 0.12 tone_friendly_2 · placebo_3 = -0.01 tone_friendly_2 · plain_1 = 0.23 tone_friendly_2 · plain_2 = 0.01 tone_friendly_2 · plain_3 = 0.02 tone_friendly_2 · ornate_1 = -0.32 tone_friendly_2 · ornate_2 = -0.17 tone_friendly_2 · ornate_3 = -0.13 tone_friendly_2 · brief_1 = 0.08 tone_friendly_2 · brief_2 = -0.18 tone_friendly_2 · brief_3 = -0.18 tone_friendly_2 · tokens_1 = -0.11 tone_friendly_2 · tokens_2 = 0.01 tone_friendly_2 · tokens_3 = -0.14 tone_friendly_2 · verbose_1 = -0.16 tone_friendly_2 · verbose_2 = -0.18 tone_friendly_2 · verbose_3 = -0.16 tone_friendly_2 · tone_formal_1 = -0.63 tone_friendly_2 · tone_formal_2 = -0.65 tone_friendly_2 · tone_formal_3 = -0.43 tone_friendly_2 · tone_friendly_1 = 0.91 tone_friendly_2 · tone_friendly_2 = 1 tone_friendly_2 · tone_friendly_3 = 0.96 tone_friendly_2 · cot_1 = -0.06 tone_friendly_2 · cot_2 = -0.12 tone_friendly_2 · cot_3 = -0.05 tone_friendly_2 · direct_1 = -0.35 tone_friendly_2 · direct_2 = -0.43 tone_friendly_2 · direct_3 = -0.27 tone_friendly_2 · careful_1 = -0.31 tone_friendly_2 · careful_2 = 0.03 tone_friendly_2 · careful_3 = -0.22 tone_friendly_2 · careless_1 = 0.12 tone_friendly_2 · careless_2 = 0.06 tone_friendly_2 · careless_3 = 0.57 tone_friendly_3 · none = 0.08 tone_friendly_3 · placebo_1 = 0.04 tone_friendly_3 · placebo_2 = 0.11 tone_friendly_3 · placebo_3 = -0.03 tone_friendly_3 · plain_1 = 0.18 tone_friendly_3 · plain_2 = -0.04 tone_friendly_3 · plain_3 = -0.03 tone_friendly_3 · ornate_1 = -0.27 tone_friendly_3 · ornate_2 = -0.13 tone_friendly_3 · ornate_3 = -0.09 tone_friendly_3 · brief_1 = 0.01 tone_friendly_3 · brief_2 = -0.26 tone_friendly_3 · brief_3 = -0.24 tone_friendly_3 · tokens_1 = -0.15 tone_friendly_3 · tokens_2 = -0.04 tone_friendly_3 · tokens_3 = -0.16 tone_friendly_3 · verbose_1 = -0.14 tone_friendly_3 · verbose_2 = -0.12 tone_friendly_3 · verbose_3 = -0.1 tone_friendly_3 · tone_formal_1 = -0.63 tone_friendly_3 · tone_formal_2 = -0.63 tone_friendly_3 · tone_formal_3 = -0.44 tone_friendly_3 · tone_friendly_1 = 0.88 tone_friendly_3 · tone_friendly_2 = 0.96 tone_friendly_3 · tone_friendly_3 = 1 tone_friendly_3 · cot_1 = -0.06 tone_friendly_3 · cot_2 = -0.07 tone_friendly_3 · cot_3 = -0.04 tone_friendly_3 · direct_1 = -0.4 tone_friendly_3 · direct_2 = -0.43 tone_friendly_3 · direct_3 = -0.33 tone_friendly_3 · careful_1 = -0.31 tone_friendly_3 · careful_2 = 0.03 tone_friendly_3 · careful_3 = -0.19 tone_friendly_3 · careless_1 = 0.17 tone_friendly_3 · careless_2 = 0.04 tone_friendly_3 · careless_3 = 0.55 cot_1 · none = 0.08 cot_1 · placebo_1 = 0.17 cot_1 · placebo_2 = 0.19 cot_1 · placebo_3 = 0.17 cot_1 · plain_1 = -0.15 cot_1 · plain_2 = -0.12 cot_1 · plain_3 = -0.07 cot_1 · ornate_1 = -0.28 cot_1 · ornate_2 = -0.27 cot_1 · ornate_3 = -0.41 cot_1 · brief_1 = -0.21 cot_1 · brief_2 = -0.23 cot_1 · brief_3 = -0.33 cot_1 · tokens_1 = -0.1 cot_1 · tokens_2 = -0.16 cot_1 · tokens_3 = -0.14 cot_1 · verbose_1 = 0.19 cot_1 · verbose_2 = 0.36 cot_1 · verbose_3 = 0.27 cot_1 · tone_formal_1 = -0.06 cot_1 · tone_formal_2 = -0.05 cot_1 · tone_formal_3 = 0.03 cot_1 · tone_friendly_1 = 0.04 cot_1 · tone_friendly_2 = -0.06 cot_1 · tone_friendly_3 = -0.06 cot_1 · cot_1 = 1 cot_1 · cot_2 = 0.57 cot_1 · cot_3 = 0.69 cot_1 · direct_1 = -0.35 cot_1 · direct_2 = -0.21 cot_1 · direct_3 = -0.32 cot_1 · careful_1 = 0.01 cot_1 · careful_2 = 0.11 cot_1 · careful_3 = 0.15 cot_1 · careless_1 = 0 cot_1 · careless_2 = 0.05 cot_1 · careless_3 = -0.06 cot_2 · none = 0.22 cot_2 · placebo_1 = 0.37 cot_2 · placebo_2 = 0.4 cot_2 · placebo_3 = 0.38 cot_2 · plain_1 = -0.3 cot_2 · plain_2 = -0.32 cot_2 · plain_3 = -0.21 cot_2 · ornate_1 = -0.27 cot_2 · ornate_2 = -0.2 cot_2 · ornate_3 = -0.38 cot_2 · brief_1 = -0.35 cot_2 · brief_2 = -0.38 cot_2 · brief_3 = -0.57 cot_2 · tokens_1 = -0.24 cot_2 · tokens_2 = -0.31 cot_2 · tokens_3 = -0.25 cot_2 · verbose_1 = 0.44 cot_2 · verbose_2 = 0.64 cot_2 · verbose_3 = 0.57 cot_2 · tone_formal_1 = -0.06 cot_2 · tone_formal_2 = 0 cot_2 · tone_formal_3 = 0.03 cot_2 · tone_friendly_1 = 0.05 cot_2 · tone_friendly_2 = -0.12 cot_2 · tone_friendly_3 = -0.07 cot_2 · cot_1 = 0.57 cot_2 · cot_2 = 1 cot_2 · cot_3 = 0.5 cot_2 · direct_1 = -0.45 cot_2 · direct_2 = -0.24 cot_2 · direct_3 = -0.47 cot_2 · careful_1 = 0.32 cot_2 · careful_2 = 0.25 cot_2 · careful_3 = 0.48 cot_2 · careless_1 = 0.19 cot_2 · careless_2 = 0.18 cot_2 · careless_3 = 0.04 cot_3 · none = 0.08 cot_3 · placebo_1 = 0.16 cot_3 · placebo_2 = 0.16 cot_3 · placebo_3 = 0.18 cot_3 · plain_1 = 0 cot_3 · plain_2 = -0.03 cot_3 · plain_3 = -0.03 cot_3 · ornate_1 = -0.37 cot_3 · ornate_2 = -0.32 cot_3 · ornate_3 = -0.44 cot_3 · brief_1 = -0.19 cot_3 · brief_2 = -0.2 cot_3 · brief_3 = -0.3 cot_3 · tokens_1 = -0.06 cot_3 · tokens_2 = -0.12 cot_3 · tokens_3 = -0.1 cot_3 · verbose_1 = 0.19 cot_3 · verbose_2 = 0.28 cot_3 · verbose_3 = 0.22 cot_3 · tone_formal_1 = -0.12 cot_3 · tone_formal_2 = -0.08 cot_3 · tone_formal_3 = -0.03 cot_3 · tone_friendly_1 = -0.01 cot_3 · tone_friendly_2 = -0.05 cot_3 · tone_friendly_3 = -0.04 cot_3 · cot_1 = 0.69 cot_3 · cot_2 = 0.5 cot_3 · cot_3 = 1 cot_3 · direct_1 = -0.3 cot_3 · direct_2 = -0.17 cot_3 · direct_3 = -0.26 cot_3 · careful_1 = 0.02 cot_3 · careful_2 = 0.05 cot_3 · careful_3 = 0.12 cot_3 · careless_1 = 0.09 cot_3 · careless_2 = 0.17 cot_3 · careless_3 = -0.02 direct_1 · none = -0.06 direct_1 · placebo_1 = -0.11 direct_1 · placebo_2 = -0.19 direct_1 · placebo_3 = -0.08 direct_1 · plain_1 = 0.2 direct_1 · plain_2 = 0.45 direct_1 · plain_3 = 0.33 direct_1 · ornate_1 = 0.02 direct_1 · ornate_2 = -0.15 direct_1 · ornate_3 = 0 direct_1 · brief_1 = 0.44 direct_1 · brief_2 = 0.58 direct_1 · brief_3 = 0.71 direct_1 · tokens_1 = 0.33 direct_1 · tokens_2 = 0.27 direct_1 · tokens_3 = 0.28 direct_1 · verbose_1 = -0.28 direct_1 · verbose_2 = -0.46 direct_1 · verbose_3 = -0.41 direct_1 · tone_formal_1 = 0.19 direct_1 · tone_formal_2 = 0.09 direct_1 · tone_formal_3 = 0.19 direct_1 · tone_friendly_1 = -0.45 direct_1 · tone_friendly_2 = -0.35 direct_1 · tone_friendly_3 = -0.4 direct_1 · cot_1 = -0.35 direct_1 · cot_2 = -0.45 direct_1 · cot_3 = -0.3 direct_1 · direct_1 = 1 direct_1 · direct_2 = 0.67 direct_1 · direct_3 = 0.85 direct_1 · careful_1 = 0.11 direct_1 · careful_2 = -0.07 direct_1 · careful_3 = -0.09 direct_1 · careless_1 = -0.3 direct_1 · careless_2 = -0.13 direct_1 · careless_3 = -0.17 direct_2 · none = -0.56 direct_2 · placebo_1 = -0.55 direct_2 · placebo_2 = -0.59 direct_2 · placebo_3 = -0.51 direct_2 · plain_1 = 0.29 direct_2 · plain_2 = 0.36 direct_2 · plain_3 = 0.1 direct_2 · ornate_1 = 0.04 direct_2 · ornate_2 = -0.07 direct_2 · ornate_3 = 0.07 direct_2 · brief_1 = 0.09 direct_2 · brief_2 = 0.28 direct_2 · brief_3 = 0.68 direct_2 · tokens_1 = 0.53 direct_2 · tokens_2 = 0.51 direct_2 · tokens_3 = 0.67 direct_2 · verbose_1 = -0.56 direct_2 · verbose_2 = -0.36 direct_2 · verbose_3 = -0.5 direct_2 · tone_formal_1 = 0.03 direct_2 · tone_formal_2 = 0.09 direct_2 · tone_formal_3 = -0.14 direct_2 · tone_friendly_1 = -0.49 direct_2 · tone_friendly_2 = -0.43 direct_2 · tone_friendly_3 = -0.43 direct_2 · cot_1 = -0.21 direct_2 · cot_2 = -0.24 direct_2 · cot_3 = -0.17 direct_2 · direct_1 = 0.67 direct_2 · direct_2 = 1 direct_2 · direct_3 = 0.78 direct_2 · careful_1 = -0.26 direct_2 · careful_2 = -0.54 direct_2 · careful_3 = -0.35 direct_2 · careless_1 = 0.16 direct_2 · careless_2 = 0.08 direct_2 · careless_3 = -0.5 direct_3 · none = -0.29 direct_3 · placebo_1 = -0.35 direct_3 · placebo_2 = -0.4 direct_3 · placebo_3 = -0.34 direct_3 · plain_1 = 0.47 direct_3 · plain_2 = 0.55 direct_3 · plain_3 = 0.3 direct_3 · ornate_1 = -0.14 direct_3 · ornate_2 = -0.22 direct_3 · ornate_3 = -0.07 direct_3 · brief_1 = 0.46 direct_3 · brief_2 = 0.59 direct_3 · brief_3 = 0.87 direct_3 · tokens_1 = 0.6 direct_3 · tokens_2 = 0.59 direct_3 · tokens_3 = 0.58 direct_3 · verbose_1 = -0.52 direct_3 · verbose_2 = -0.59 direct_3 · verbose_3 = -0.62 direct_3 · tone_formal_1 = -0.03 direct_3 · tone_formal_2 = -0.08 direct_3 · tone_formal_3 = -0.07 direct_3 · tone_friendly_1 = -0.36 direct_3 · tone_friendly_2 = -0.27 direct_3 · tone_friendly_3 = -0.33 direct_3 · cot_1 = -0.32 direct_3 · cot_2 = -0.47 direct_3 · cot_3 = -0.26 direct_3 · direct_1 = 0.85 direct_3 · direct_2 = 0.78 direct_3 · direct_3 = 1 direct_3 · careful_1 = -0.13 direct_3 · careful_2 = -0.35 direct_3 · careful_3 = -0.29 direct_3 · careless_1 = -0.11 direct_3 · careless_2 = 0.13 direct_3 · careless_3 = -0.25 careful_1 · none = 0.77 careful_1 · placebo_1 = 0.78 careful_1 · placebo_2 = 0.73 careful_1 · placebo_3 = 0.82 careful_1 · plain_1 = -0.25 careful_1 · plain_2 = 0.01 careful_1 · plain_3 = 0.28 careful_1 · ornate_1 = -0.2 careful_1 · ornate_2 = -0.32 careful_1 · ornate_3 = -0.35 careful_1 · brief_1 = 0.23 careful_1 · brief_2 = 0.24 careful_1 · brief_3 = -0.27 careful_1 · tokens_1 = -0.3 careful_1 · tokens_2 = -0.44 careful_1 · tokens_3 = -0.5 careful_1 · verbose_1 = 0.73 careful_1 · verbose_2 = 0.39 careful_1 · verbose_3 = 0.59 careful_1 · tone_formal_1 = 0.35 careful_1 · tone_formal_2 = 0.21 careful_1 · tone_formal_3 = 0.59 careful_1 · tone_friendly_1 = -0.27 careful_1 · tone_friendly_2 = -0.31 careful_1 · tone_friendly_3 = -0.31 careful_1 · cot_1 = 0.01 careful_1 · cot_2 = 0.32 careful_1 · cot_3 = 0.02 careful_1 · direct_1 = 0.11 careful_1 · direct_2 = -0.26 careful_1 · direct_3 = -0.13 careful_1 · careful_1 = 1 careful_1 · careful_2 = 0.79 careful_1 · careful_3 = 0.88 careful_1 · careless_1 = -0.33 careful_1 · careless_2 = 0.03 careful_1 · careless_3 = 0.33 careful_2 · none = 0.9 careful_2 · placebo_1 = 0.9 careful_2 · placebo_2 = 0.86 careful_2 · placebo_3 = 0.86 careful_2 · plain_1 = -0.36 careful_2 · plain_2 = -0.06 careful_2 · plain_3 = 0.3 careful_2 · ornate_1 = -0.18 careful_2 · ornate_2 = -0.31 careful_2 · ornate_3 = -0.32 careful_2 · brief_1 = 0.28 careful_2 · brief_2 = 0.17 careful_2 · brief_3 = -0.37 careful_2 · tokens_1 = -0.42 careful_2 · tokens_2 = -0.52 careful_2 · tokens_3 = -0.65 careful_2 · verbose_1 = 0.7 careful_2 · verbose_2 = 0.32 careful_2 · verbose_3 = 0.61 careful_2 · tone_formal_1 = 0.2 careful_2 · tone_formal_2 = -0.01 careful_2 · tone_formal_3 = 0.54 careful_2 · tone_friendly_1 = 0.07 careful_2 · tone_friendly_2 = 0.03 careful_2 · tone_friendly_3 = 0.03 careful_2 · cot_1 = 0.11 careful_2 · cot_2 = 0.25 careful_2 · cot_3 = 0.05 careful_2 · direct_1 = -0.07 careful_2 · direct_2 = -0.54 careful_2 · direct_3 = -0.35 careful_2 · careful_1 = 0.79 careful_2 · careful_2 = 1 careful_2 · careful_3 = 0.78 careful_2 · careless_1 = -0.46 careful_2 · careless_2 = -0.18 careful_2 · careless_3 = 0.53 careful_3 · none = 0.75 careful_3 · placebo_1 = 0.78 careful_3 · placebo_2 = 0.76 careful_3 · placebo_3 = 0.79 careful_3 · plain_1 = -0.27 careful_3 · plain_2 = -0.1 careful_3 · plain_3 = 0.17 careful_3 · ornate_1 = -0.27 careful_3 · ornate_2 = -0.34 careful_3 · ornate_3 = -0.41 careful_3 · brief_1 = 0.14 careful_3 · brief_2 = 0.12 careful_3 · brief_3 = -0.4 careful_3 · tokens_1 = -0.32 careful_3 · tokens_2 = -0.43 careful_3 · tokens_3 = -0.5 careful_3 · verbose_1 = 0.74 careful_3 · verbose_2 = 0.47 careful_3 · verbose_3 = 0.67 careful_3 · tone_formal_1 = 0.23 careful_3 · tone_formal_2 = 0.11 careful_3 · tone_formal_3 = 0.46 careful_3 · tone_friendly_1 = -0.14 careful_3 · tone_friendly_2 = -0.22 careful_3 · tone_friendly_3 = -0.19 careful_3 · cot_1 = 0.15 careful_3 · cot_2 = 0.48 careful_3 · cot_3 = 0.12 careful_3 · direct_1 = -0.09 careful_3 · direct_2 = -0.35 careful_3 · direct_3 = -0.29 careful_3 · careful_1 = 0.88 careful_3 · careful_2 = 0.78 careful_3 · careful_3 = 1 careful_3 · careless_1 = -0.18 careful_3 · careless_2 = 0.11 careful_3 · careless_3 = 0.37 careless_1 · none = -0.38 careless_1 · placebo_1 = -0.36 careless_1 · placebo_2 = -0.31 careless_1 · placebo_3 = -0.33 careless_1 · plain_1 = 0.22 careless_1 · plain_2 = -0.11 careless_1 · plain_3 = -0.36 careless_1 · ornate_1 = -0.17 careless_1 · ornate_2 = 0.03 careless_1 · ornate_3 = 0.03 careless_1 · brief_1 = -0.38 careless_1 · brief_2 = -0.4 careless_1 · brief_3 = -0.18 careless_1 · tokens_1 = 0.17 careless_1 · tokens_2 = 0.26 careless_1 · tokens_3 = 0.35 careless_1 · verbose_1 = -0.26 careless_1 · verbose_2 = 0.05 careless_1 · verbose_3 = -0.1 careless_1 · tone_formal_1 = -0.43 careless_1 · tone_formal_2 = -0.23 careless_1 · tone_formal_3 = -0.6 careless_1 · tone_friendly_1 = 0.13 careless_1 · tone_friendly_2 = 0.12 careless_1 · tone_friendly_3 = 0.17 careless_1 · cot_1 = 0 careless_1 · cot_2 = 0.19 careless_1 · cot_3 = 0.09 careless_1 · direct_1 = -0.3 careless_1 · direct_2 = 0.16 careless_1 · direct_3 = -0.11 careless_1 · careful_1 = -0.33 careless_1 · careful_2 = -0.46 careless_1 · careful_3 = -0.18 careless_1 · careless_1 = 1 careless_1 · careless_2 = 0.33 careless_1 · careless_3 = -0.04 careless_2 · none = 0.04 careless_2 · placebo_1 = 0.05 careless_2 · placebo_2 = 0.07 careless_2 · placebo_3 = 0.05 careless_2 · plain_1 = 0.52 careless_2 · plain_2 = 0.1 careless_2 · plain_3 = 0.01 careless_2 · ornate_1 = -0.6 careless_2 · ornate_2 = -0.39 careless_2 · ornate_3 = -0.37 careless_2 · brief_1 = 0.19 careless_2 · brief_2 = 0.15 careless_2 · brief_3 = 0.07 careless_2 · tokens_1 = 0.36 careless_2 · tokens_2 = 0.37 careless_2 · tokens_3 = 0.25 careless_2 · verbose_1 = 0.04 careless_2 · verbose_2 = 0.06 careless_2 · verbose_3 = -0.01 careless_2 · tone_formal_1 = -0.45 careless_2 · tone_formal_2 = -0.38 careless_2 · tone_formal_3 = -0.37 careless_2 · tone_friendly_1 = 0.05 careless_2 · tone_friendly_2 = 0.06 careless_2 · tone_friendly_3 = 0.04 careless_2 · cot_1 = 0.05 careless_2 · cot_2 = 0.18 careless_2 · cot_3 = 0.17 careless_2 · direct_1 = -0.13 careless_2 · direct_2 = 0.08 careless_2 · direct_3 = 0.13 careless_2 · careful_1 = 0.03 careless_2 · careful_2 = -0.18 careless_2 · careful_3 = 0.11 careless_2 · careless_1 = 0.33 careless_2 · careless_2 = 1 careless_2 · careless_3 = 0.28 careless_3 · none = 0.66 careless_3 · placebo_1 = 0.62 careless_3 · placebo_2 = 0.63 careless_3 · placebo_3 = 0.56 careless_3 · plain_1 = 0.16 careless_3 · plain_2 = -0.02 careless_3 · plain_3 = 0.16 careless_3 · ornate_1 = -0.51 careless_3 · ornate_2 = -0.37 careless_3 · ornate_3 = -0.3 careless_3 · brief_1 = 0.35 careless_3 · brief_2 = 0.09 careless_3 · brief_3 = -0.27 careless_3 · tokens_1 = -0.2 careless_3 · tokens_2 = -0.15 careless_3 · tokens_3 = -0.38 careless_3 · verbose_1 = 0.35 careless_3 · verbose_2 = 0.09 careless_3 · verbose_3 = 0.24 careless_3 · tone_formal_1 = -0.5 careless_3 · tone_formal_2 = -0.59 careless_3 · tone_formal_3 = -0.17 careless_3 · tone_friendly_1 = 0.54 careless_3 · tone_friendly_2 = 0.57 careless_3 · tone_friendly_3 = 0.55 careless_3 · cot_1 = -0.06 careless_3 · cot_2 = 0.04 careless_3 · cot_3 = -0.02 careless_3 · direct_1 = -0.17 careless_3 · direct_2 = -0.5 careless_3 · direct_3 = -0.25 careless_3 · careful_1 = 0.33 careless_3 · careful_2 = 0.53 careless_3 · careful_3 = 0.37 careless_3 · careless_1 = -0.04 careless_3 · careless_2 = 0.28 careless_3 · careless_3 = 1 none none placebo_1 placebo_1 placebo_2 placebo_2 placebo_3 placebo_3 plain_1 plain_1 plain_2 plain_2 plain_3 plain_3 ornate_1 ornate_1 ornate_2 ornate_2 ornate_3 ornate_3 brief_1 brief_1 brief_2 brief_2 brief_3 brief_3 tokens_1 tokens_1 tokens_2 tokens_2 tokens_3 tokens_3 verbose_1 verbose_1 verbose_2 verbose_2 verbose_3 verbose_3 tone_formal_1 tone_formal_1 tone_formal_2 tone_formal_2 tone_formal_3 tone_formal_3 tone_friendly_1 tone_friendly_1 tone_friendly_2 tone_friendly_2 tone_friendly_3 tone_friendly_3 cot_1 cot_1 cot_2 cot_2 cot_3 cot_3 direct_1 direct_1 direct_2 direct_2 direct_3 direct_3 careful_1 careful_1 careful_2 careful_2 careful_3 careful_3 careless_1 careless_1 careless_2 careless_2 careless_3 careless_3 -1 0 1
Some fun observations pop out of this.
First, we can see clearly that despite our best efforts in creating the style prompts, they do not perfectly correlate with each other within a cluster! Also, we can see that the opposite sides of our style prompt axes (brief - verbose, plain - ornate) are opposed. It’s a known result in transformer interp that “opposite” prompts are not necessarily mathematically opposite vectors in state space.
The positive and negative controls are pretty close to the verbose cluster. This makes sense because LLMs are pretty verbose by default, but imo it’s still an interesting result to see that quantitatively. We can also see that the careful cluster is really close to the placebo cluster, so we know they really hammered that into Gemma’s default behavior. Also, careless_3 aligns better with none and even careful than it does with the other careless prompts.
Finally, we can see here that avoid mannered prose and keep it brief are only weakly aligned. This means that they mean different things to the model! I have wondered if I need to use both for Claude, and this indicates to me that yes, I may need to type the extra 3 words with each new prompt.
Pairwise heatmaps can be hard to read, so Figure 5 shows a 2d projection via multidimensional scaling (MDS) on that pairwise cosine matrix. More aligned prompts are closer together. Hover to show the actual style prompt text.
-0.5 0 0.5 1 -1 -0.5 0 0.5 none (no style prompt) none placebo_1 — “Answer the request below.” placebo_1 placebo_2 — “Respond to the following request.” placebo_2 placebo_3 — “Please complete the task below.” placebo_3 plain_1 — “Avoid mannered prose.” plain_1 plain_2 — “Write plainly, without affectation.” plain_2 plain_3 — “Avoid purple prose.” plain_3 ornate_1 — “Use mannered prose.” ornate_1 ornate_2 — “Write ornately, with affectation.” ornate_2 ornate_3 — “Use purple prose.” ornate_3 brief_1 — “Keep it brief.” brief_1 brief_2 — “Be concise.” brief_2 brief_3 — “Use as few words as needed.” brief_3 tokens_1 — “Minimize output tokens.” tokens_1 tokens_2 — “Minimize your token count.” tokens_2 tokens_3 — “Output the fewest tokens you can.” tokens_3 verbose_1 — “Be thorough and detailed.” verbose_1 verbose_2 — “Explain at length.” verbose_2 verbose_3 — “Answer in depth.” verbose_3 tone_formal_1 — “Use a formal tone.” tone_formal_1 tone_formal_2 — “Write in a formal register.” tone_formal_2 tone_formal_3 — “Maintain a professional tone.” tone_formal_3 tone_friendly_1 — “Write in a friendly tone.” tone_friendly_1 tone_friendly_2 — “Use a warm, casual tone.” tone_friendly_2 tone_friendly_3 — “Keep it warm and conversational.” tone_friendly_3 cot_1 — “Please explain your reasoning first.” cot_1 cot_2 — “Please show how you got your answer.” cot_2 cot_3 — “Please write out your chain of thought first.” cot_3 direct_1 — “Please answer without explaining your reasoning.” direct_1 direct_2 — “Please give just the answer, not how you got it.” direct_2 direct_3 — “Please answer directly, without any chain of thought.” direct_3 careful_1 — “Make no mistakes.” careful_1 careful_2 — “Answer carefully.” careful_2 careful_3 — “Be certain of your correctness.” careful_3 careless_1 — “Make mistakes.” careless_1 careless_2 — “Prioritize speed over precision.” careless_2 careless_3 — “Don't worry about being correct.” careless_3 PC1 PC2
This view really makes some of our within-cluster divergences clear. Why is tone_formal_3 (“Maintain a professional tone”) so far from the other two tone_formal prompts? Why are the three brief style prompts and the three careless style prompts scattered so far apart? Clearly, prompt modifiers that seem close in meaning to human can have wildly different effects on a model and its outputs.
How consistent is a style prompt’s effect across different task types?
Let’s define style prompt “consistency”. For a given style prompt \(j\), take its per-base-prompt effect vectors as \(d_i\) where \(i \in [0, 538]\). Then, take \(m = \operatorname{mean}_i(d_i)\) and show each \(d_i\) as \(d_i = m + e_i\). Then, take consistency for prompt \(j\) to be
\(\rho_j = \frac{\lVert m \rVert^2}{\operatorname{mean}_i \lVert d_i \rVert^2}.\)
If this ratio is high, then we know that the mean effect is most of the individual per-prompt effect, which means that style prompt is consistent across a range of tasks. Figure 6 shows this ratio for each of our style prompt clusters in both the internal state and the first-token logit distribution.
0 0.2 0.4 0.6 direct: rho = 0.123 direct careful: rho = 0.135 careful brief: rho = 0.153 brief placebo: rho = 0.183 placebo verbose: rho = 0.197 verbose tokens: rho = 0.211 tokens plain: rho = 0.244 plain careless: rho = 0.254 careless cot: rho = 0.274 cot tone_formal: rho = 0.34 tone_formal tone_friendly: rho = 0.437 tone_friendly ornate: rho = 0.55 ornate consistency ρ = ‖mean effect‖² / mean‖effect‖² (final layer)
This result is super interesting to me. Honestly, this is not what I expected to see here. My hypotheses-interpretation here is ornate/purple/”mannered” outputs are a lower information density and they have more fluff that will be ~the same fluff no matter the context, meaning the outputs will appear more consistent.
Let’s take it a step further. I categorized each of the 538 base prompts into 6 groups. Figure 7 shows our plain cluster’s effect consistency between task types.
0 0.1 0.2 0.3 0.4 0.5 Closed QA: rho = 0.217 [0.191, 0.282], n=45 Closed QA n=45 Rewrite: rho = 0.244 [0.224, 0.303], n=45 Rewrite n=45 Generation: rho = 0.277 [0.267, 0.293], n=333 Generation n=333 Open QA: rho = 0.318 [0.29, 0.367], n=69 Open QA n=69 Summarization: rho = 0.344 [0.322, 0.461], n=15 Summarization n=15 Brainstorming: rho = 0.353 [0.323, 0.437], n=31 Brainstorming n=31 consistency ρ of the plain (avoid-mannered) effect, by task cluster
This figure shows the plain cluster’s effect across task types. Most of the prompts are in the large Generation category, so the pooled ρ is biased heavily towards that category. We could shore this up with a more evenly distributed task set.
Do prompts like Avoid Mannered Prose work?
We could assess style subjectively over a set of prompts, but I prefer to have some computational measurement so we can compare all outputs we generated. Luckily, there are a wealth of “stylometrics” we can use to assess these prompts’ effects on readability. Figure 8 shows the distribution of each stylometric for one representative style prompt per cluster over the full set of 539 base prompts as CDFs or cumulative fractions. I capped max_new_tokens = 128 for these generation runs.
0 0.5 1 0 50 100 Flesch reading ease (higher=plainer) none placebo plain ornate brief tokens verbose tone_formal tone_friendly cot direct careful careless 0 0.5 1 1.5 2 syllables / word none placebo plain ornate brief tokens verbose tone_formal tone_friendly cot direct careful careless 0 0.5 1 3 4 5 6 characters / word none placebo plain ornate brief tokens verbose tone_formal tone_friendly cot direct careful careless 0 0.5 1 0 20 40 % polysyllabic (>=3 syl) none placebo plain ornate brief tokens verbose tone_formal tone_friendly cot direct careful careless 0 0.5 1 0 10 20 commas per 100 words none placebo plain ornate brief tokens verbose tone_formal tone_friendly cot direct careful careless 0 0.5 1 2 4 6 Guiraud lexical richness none placebo plain ornate brief tokens verbose tone_formal tone_friendly cot direct careful careless 0 0.5 1 50 100 word count none placebo plain ornate brief tokens verbose tone_formal tone_friendly cot direct careful careless cumulative fraction on y
Our plain, brief, and direct clusters are consistently on the easier-to-read side of these distributions. The tone_friendly cluster is also frequently easier to read. The harder-to-read ends of these distribution are more variable.
Some interesting observations here:
tone_friendly beats plain on every readability metric!careful is usually right on top of placeboverbose has higher overall word count than placebo but is otherwise right next to itverbose does not have the highest word count. Both verbose and placebo have right-shifted distributions
Figure 9 shows how these measurements vary between task types for just our plain cluster.
0 0.5 1 0 50 100 Flesch reading ease (higher=plainer) Generation Open QA Closed QA Rewrite Brainstorming Summarization 0 0.5 1 1.5 2 syllables / word Generation Open QA Closed QA Rewrite Brainstorming Summarization 0 0.5 1 4 5 6 characters / word Generation Open QA Closed QA Rewrite Brainstorming Summarization 0 0.5 1 0 20 40 % polysyllabic (>=3 syl) Generation Open QA Closed QA Rewrite Brainstorming Summarization 0 0.5 1 0 10 20 commas per 100 words Generation Open QA Closed QA Rewrite Brainstorming Summarization 0 0.5 1 4 6 Guiraud lexical richness Generation Open QA Closed QA Rewrite Brainstorming Summarization 0 0.5 1 50 100 word count Generation Open QA Closed QA Rewrite Brainstorming Summarization cumulative fraction on y
It’s pretty clear here that task type has a strong influence on these metrics. The relative positions of these distributions are roughly what you would expect from the task type. In general, diction features are stable across task types, but syntax features are determined by the specific task. That across-task variability is the source of the plain cluster variance in Figure 7.
I tried to replicate these results by using a set of steering vectors instead of the style prompt, but the results were not as consistent or helpful as just using the style prompt itself. More to come on that later.
Conclusion
Clearly, these style prompts are shifting the model’s behavior, but sometimes the results are unpredictable. This analysis shows how “prompt engineering” conducted manually with individual prompts can be extremely brittle. To improve our work with these models, we need to better understand how prompt changes affect outputs across task types. A model’s documentation should include quantitative specs for the effects of style prompts and other prompting mechanisms just like manufacturer data sheets show force/deformation curves for springs and other mechanical components. That knowledge would allow us to use LLMs more effectively.
