Binary arithmetic | Floating point types

Bfloat16 floating-point format

The bfloat16 (Brain Floating Point) floating-point format is a computer number format occupying 16 bits in computer memory; it represents a wide dynamic range of numeric values by using a floating radix point. This format is a truncated (16-bit) version of the 32-bit IEEE 754 single-precision floating-point format (binary32) with the intent of accelerating machine learning and near-sensor computing. It preserves the approximate dynamic range of 32-bit floating-point numbers by retaining 8 exponent bits, but supports only an 8-bit precision rather than the 24-bit significand of the binary32 format. More so than single-precision 32-bit floating-point numbers, bfloat16 numbers are unsuitable for integer calculations, but this is not their intended use. Bfloat16 is used to reduce the storage requirements and increase the calculation speed of machine learning algorithms. The bfloat16 format was developed by Google Brain, an artificial intelligence research group at Google. The bfloat16 format is utilized in Intel AI processors, such as Nervana NNP-L1000, Xeon processors (AVX-512 BF16 extensions), and Intel FPGAs, Google Cloud TPUs, and TensorFlow. ARMv8.6-A, AMD ROCm, and CUDA also support the bfloat16 format. On these platforms, bfloat16 may also be used in mixed-precision arithmetic, where bfloat16 numbers may be operated on and expanded to wider data types. (Wikipedia).

Video thumbnail

TPUs, systolic arrays, and bfloat16: accelerate your deep learning | Kaggle

Today we’re going to talk about systolic arrays and bfloat16 multipliers, two components of tensor processing units (TPUs) that are responsible for accelerating your deep learning model training time. ** super important content note **: you may have caught that the on screen image at 4:4

From playlist Deep Learning | Kaggle

Video thumbnail

Let's make 16 games in C++: Xonix

Download source: https://drive.google.com/uc?export=download&id=1X24AF6OYBp0dFDdjtTx0nlTrGOHb4uRr

From playlist Let's make 16 games in C++/SFML!

Video thumbnail

Let's make 16 games in C++: Blobby Volley

Download source: https://drive.google.com/uc?export=download&id=1X24AF6OYBp0dFDdjtTx0nlTrGOHb4uRr C++ / SFML / Box2d example

From playlist Let's make 16 games in C++/SFML!

Video thumbnail

Let's make 16 games in C++: Chess

Download source: https://drive.google.com/uc?export=download&id=1X24AF6OYBp0dFDdjtTx0nlTrGOHb4uRr

From playlist Let's make 16 games in C++/SFML!

Video thumbnail

Let's make 16 games in C++: Bejeweled (Match-3)

Download source: https://drive.google.com/uc?export=download&id=1X24AF6OYBp0dFDdjtTx0nlTrGOHb4uRr

From playlist Let's make 16 games in C++/SFML!

Video thumbnail

Boost Fine-Tuning Performance of LLM: Optimal Architecture w/ PEFT LoRA Adapter-Tuning on Your GPU

Not enough memory to fine-tune your Language Model: T5, GPT, OPT, BLOOM, Llama, ..? Optimize your model architecture to the MAX for optimal fine-tuning (adapter-tuning) for faster, cheaper and MIN memory on your GPU! LLM Fine-Tuning on a Budget: Supercharge Your Language Model on a Norma

From playlist ALPACA and Adapter-tuning w/ PEFT & LoRA for new LLMs

Video thumbnail

Ultimate Guide To Scaling ML Models - Megatron-LM | ZeRO | DeepSpeed | Mixed Precision

πŸš€ Sign up for AssemblyAI's speech API using my link πŸš€ https://www.assemblyai.com/?utm_source=youtube&utm_medium=social&utm_campaign=theaiepiphany πŸ‘¨β€πŸ‘©β€πŸ‘§β€πŸ‘¦ Join our Discord community πŸ‘¨β€πŸ‘©β€πŸ‘§β€πŸ‘¦ https://discord.gg/peBrCpheKE In this video I show you what it takes to scale ML models up to tril

From playlist Miscellaneous

Video thumbnail

Let's make 16 games in C++: Doodle Jump

Download source: https://drive.google.com/uc?export=download&id=1X24AF6OYBp0dFDdjtTx0nlTrGOHb4uRr

From playlist Let's make 16 games in C++/SFML!

Video thumbnail

Gigabyte R292-4S1 Review w/ 1kW of Intel Xeon Cooper Lake

STH Main Site Article: https://www.servethehome.com/gigabyte-r292-4s1-server-review-2u-4-socket-cooper-lake/ STH Merch on (Tee)Spring: https://the-sth-merch-shop.myteespring.co/ STH Top 5 Weekly Newsletter: https://eepurl.com/dryM09 In our Gigabyte R292-4S1 review, we see how this server

From playlist Cool Server Hardware Series

Video thumbnail

Let's make 16 games in C++: Minesweeper

Download source: https://drive.google.com/uc?export=download&id=1X24AF6OYBp0dFDdjtTx0nlTrGOHb4uRr

From playlist Let's make 16 games in C++/SFML!

Video thumbnail

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

#ai #technology #switchtransformer Scale is the next frontier for AI. Google Brain uses sparsity and hard routing to massively increase a model's parameters, while keeping the FLOPs per forward pass constant. The Switch Transformer compares favorably to its dense counterparts in terms of

From playlist Papers Explained

Video thumbnail

Binary 4 – Floating Point Binary Fractions 1

This is the fourth in a series of videos about the binary number system which is fundamental to the operation of a digital electronic computer. In particular, this video covers the representation of real numbers using floating point binary notation. It begins with a description of standard

From playlist Binary

Video thumbnail

Hands-on with a Supermicro 4P Intel Cooper Lake Xeon Server

See the accompanying STH main site article here: https://www.servethehome.com/supermicro-sys-240p-tnrt-4p-3rd-gen-xeon-scalable-cooper-lake/ We had the opportunity to get some hands-on time with the Supermicro SYS-240P-TNRT 4P server ahead of today's Cooper Lake/ Cedar Island 3rd Generati

From playlist Cool Server Hardware Series

Video thumbnail

Let's make 16 games in C++: 15-Puzzle

Download source: https://drive.google.com/uc?export=download&id=1X24AF6OYBp0dFDdjtTx0nlTrGOHb4uRr

From playlist Let's make 16 games in C++/SFML!

Video thumbnail

How to use decimal points

πŸ‘‰ Learn all about decimals. Decimals are numbers written with a decimal point. Digits can be written to the right or to the left of the decimal point. Digits are written to the left of the decimal point increase in value by multiples of 10 while digits written to the right decrease by mul

From playlist Decimals | Learn About

Video thumbnail

IEEE 754 Standard for Floating Point Binary Arithmetic

This computer science video describes the IEEE 754 standard for floating point binary. The layouts of single precision, double precision and quadruple precision floating point binary numbers are described, including the sign bit, the biased exponent and the mantissa. Examples of how to con

From playlist Binary

Video thumbnail

Vlog #002: asm, printf and a simple bug

This time I received a short assembly (nasm/win32) program in two versions. A long one that worked and a short one that didn't. The bug turned out to be pretty simple. P.S. This is a fixed re-uploaded version. Thanks to blamedrop for spotting the issue!

From playlist Gynvael's Vlog

Video thumbnail

Binary 5 – Floating Point Range versus Precision

This is the fifth in a series of videos about the binary number system which is fundamental to the operation of a digital electronic computer. In particular, this video elaborates on the representation of real numbers using floating point binary notation. It explains how the relative allo

From playlist Binary

Video thumbnail

Let's make 16 games in C++: Arkanoid

Download source: https://drive.google.com/uc?export=download&id=1X24AF6OYBp0dFDdjtTx0nlTrGOHb4uRr

From playlist Let's make 16 games in C++/SFML!

Related pages

Binary number | IEEE 754 | CUDA | Hexadecimal | Offset binary | Single-precision floating-point format | Half-precision floating-point format | Significand | Type conversion | TensorFlow | Primitive data type | Dynamic range | Minifloat | Subnormal number | Exponent bias | OpenCL | Sign bit | Mixed-precision arithmetic | Infinity | NaN | Computer number format | ISO/IEC 10967