Discover gists

Naming Classes Without a 'Manager'

Reinforcement Learning for Language Models

Yoav Goldberg, April 2023.

Why RL?

With the release of the ChatGPT model and followup large language models (LLMs), there was a lot of discussion of the importance of "RLHF training", that is, "reinforcement learning from human feedback". I was puzzled for a while as to why RL (Reinforcement Learning) is better than learning from demonstrations (a.k.a supervised learning) for training language models. Shouldn't learning from demonstrations (or, in language model terminology "instruction fine tuning", learning to immitate human written answers) be sufficient? I came up with a theoretical argument that was somewhat convincing. But I came to realize there is an additional argumment which not only supports the case of RL training, but also requires it, in particular for models like ChatGPT. This additional argument is spelled out in (the first half of) a talk by John Schulman from OpenAI. This post pretty much

Ghostscript PDF compression on Windows

Installation

Download and install Ghostscript for windows (http://downloads.ghostscript.com/public/gs916w32.exe)
Optional - Add the ghostscript directory to the path environment variable
- Control Panel > System > Advanced System Settings > Environment Variables
- Add ;C:\Program Files (x86)\gs\gs9.16\bin to th end of the PATH variable

Usage

2004 (all are available)

Available	Heading	Update URL	Release Date	Release Version
✓	June 18, 2020—KB4567523 (OS Build 19041.331)	KB4567523	2020-06-18	OS Build 19041.331
✓	June 9, 2020—KB4557957 (OS Build 19041.329)	KB4557957	2020-06-09	OS Build 19041.329

1909

Same as 1903 below.

	import java.io.*;
	import java.util.*;

	public class BRC {
	public static void main(String[] args) throws FileNotFoundException {
	long millis = System.currentTimeMillis();
	System.out.println(calculateMinMeanMaxPerStation(readCSV()));
	System.out.println("Time taken = " + (System.currentTimeMillis() - millis));
	}

	{
	"editor.wordWrap": "on", //quebra linha automaticamente, eliminando o scroll horizontal
	"debug.disassemblyView.showSourceCode": false, //código-fonte não será mostrado na visualização de desmontagem (disassembly view) durante a depuração.
	"editor.fontFamily": "JetBrains Mono", //requer a fonte instalada na máquina
	"editor.fontSize": 12.5, //tamanho da fonte
	"editor.lineHeight": 1.8, //line height da fonte (melhora a visualização)
	"editor.renderLineHighlight": "gutter", //quando uma linha é selecionada, a borda só aparece no número da linha
	"editor.fontLigatures": true, // ligações de fonte (exemplo: => vira uma seta)
	"workbench.editor.labelFormat": "short", //deixa a label que mostra o arquivo selecionado mais clean
	"explorer.compactFolders": false, //quando uma pasta com um arquivo está dentro de outra pasta, não mostra tudo em uma linha só

-# delete local tag '12345'
-git tag -d 12345
-# delete remote tag '12345' (eg, GitHub version too)
-git push origin :refs/tags/12345
-# alternative approach
-git push --delete origin tagName
-git tag -d tagName

	import base64
	import json
	import urllib
	import requests

	org_name = "your org"
	project_name = "your project"
	repo_name = "test"
	PAT = "a_pat"

	/** Emulate `cmd1 \| cmd2 \| more` pipeline using recursion.

	http://stackoverflow.com/questions/20434124/recursive-piping-in-unix-environment
	*/
	#include <signal.h>
	#include <stdio.h>
	#include <stdlib.h>

	#include <unistd.h>

	<?xml version="1.0" encoding="UTF-8"?>
	<scriptfile>
	<settings program="actiona" version="3.10.1" scriptVersion="1.1.0" os="GNU/Linux"/>
	<actions>
	<action name="ActionClick" version="1.0.0"/>
	<action name="ActionGoto" version="1.0.0"/>
	<action name="ActionKey" version="1.0.0"/>
	<action name="ActionKeyboardKeyCondition" version="1.0.0"/>
	</actions>
	<parameters/>