Discover gists

Which GGUF is right for me? (Opinionated)

Good question! I am collecting human data on how quantization affects outputs. See here for more information: ggerganov/llama.cpp#5962

In the meantime, use the largest that fully fits in your GPU. If you can comfortably fit Q4_K_S, try using a model with more parameters.

When executing Python natively:

$ python pystone.py
Pystone(1.1) time for 50000 passes = 0.129016
This machine benchmarks at 387549 pystones/second

All sizes in MB, change /1024/1024 to /1024/1024/1024 for GB.

Without system databases

select
datname as database,
(pg_database_size(datname)/1024/1024) as size

	brew install caddy mkcert nss dnsmasq

	mkcert -install

	mkcert '.app.test' '.cdn.test'

	# rename the certs and move them under /usr/local/etc/caddy/certs

	cat <<EOF > /usr/local/etc/caddy/Caddyfile
	.app.test:443, .cdn.test:443 {

	library(geojsonsf)
	library(sf)
	library(rayrender)

	#Data source: https://github.com/telegeography/www.submarinecablemap.com
	cables = geojson_sf("cable-geo.json")

	cablescene = list()
	counter = 1
	for(i in 1:length(cables$geometry)) {

	## From a post on the ML, apropos this:
	## http://lowlatencyweb.wordpress.com/2012/03/20/500000-requestssec-modern-http-servers-are-fast.
	## For sysctl.conf

	net.ipv4.tcp_slow_start_after_idle = 0


	echo "1768 64512" > /proc/sys/net/ipv4/ip_local_port_range
	echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
	echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse

	package main


	// Working example: https://goplay.space/#Sa7qCLm6w65
	import (
	"bytes"
	"crypto/aes"
	"crypto/cipher"
	"encoding/hex"
	"fmt"

	#!/usr/bin/env bash

	CIPHERS='ALL:eNULL'
	DELAY=${2:-0.1}
	SERVER=${1:?usage: $0 <host:port> [delay, default is ${DELAY}s] [ciphers, default is ${CIPHERS}]}
	MAXLEN=$(openssl ciphers "$CIPHERS" \| sed -e 's/:/\n/g' \| awk '{ if ( length > L ) { L=length} }END{ print L}')

	echo Using $(openssl version).

	declare -A TLSMAP=( [tls1_1]=cipher [tls1_2]=cipher [tls1_3]=ciphersuites )

	@Inherited
	@InterceptorBinding
	@Retention(RetentionPolicy.RUNTIME)
	@Target({ElementType.METHOD,ElementType.TYPE})
	public @interface Lock {
	LockType value() default LockType.WRITE;
	}

	-- show running queries (pre 9.2)
	SELECT procpid, age(clock_timestamp(), query_start), usename, current_query
	FROM pg_stat_activity
	WHERE current_query != '<IDLE>' AND current_query NOT ILIKE '%pg_stat_activity%'
	ORDER BY query_start desc;

	-- show running queries (9.2)
	SELECT pid, age(clock_timestamp(), query_start), usename, query
	FROM pg_stat_activity
	WHERE query != '<IDLE>' AND query NOT ILIKE '%pg_stat_activity%'