From 07c56086aa0c4e015c9044e333ae4001debcb28d Mon Sep 17 00:00:00 2001 From: Paul Buetow Date: Tue, 30 Apr 2024 13:14:09 +0300 Subject: Update content for html --- gemfeed/2008-06-26-perl-poetry.html | 10 +- ...12-29-using-my-nokia-n95-for-fixing-my-mta.html | 2 +- gemfeed/2010-04-09-standard-ml-and-haskell.html | 14 +- ...010-05-07-lazy-evaluation-with-standarn-ml.html | 6 +- .../2010-05-09-the-fype-programming-language.html | 50 +- .../2011-05-07-perl-daemon-service-framework.html | 20 +- .../2014-03-24-the-fibonacci.pl.c-polyglot.html | 8 +- ...2-05-run-debian-on-your-phone-with-debroid.html | 18 +- gemfeed/2016-04-03-offsite-backup-with-zfs.html | 10 +- ...04-09-jails-and-zfs-on-freebsd-with-puppet.html | 10 +- .../2016-04-16-offsite-backup-with-zfs-part2.html | 2 +- ...inning-up-my-own-authoritative-dns-servers.html | 16 +- ...20-object-oriented-programming-with-ansi-c.html | 10 +- ...alistic-load-testing-with-ioriot-for-linux.html | 28 +- ...-22-dtail-the-distributed-log-tail-program.html | 20 +- gemfeed/2021-04-24-welcome-to-the-geminispace.html | 16 +- ...021-05-16-personal-bash-coding-style-guide.html | 36 +- ...5-gemtexter-one-bash-script-to-rule-it-all.html | 26 +- gemfeed/2021-07-04-the-well-grounded-rubyist.html | 22 +- ...-08-01-on-being-pedantic-about-open-source.html | 30 +- gemfeed/2021-09-12-keep-it-simple-and-stupid.html | 28 +- gemfeed/2021-10-22-defensive-devops.html | 18 +- gemfeed/2021-11-29-bash-golf-part-1.html | 18 +- ...-12-26-how-to-stay-sane-as-a-devops-person.html | 30 +- gemfeed/2022-01-01-bash-golf-part-2.html | 16 +- gemfeed/2022-01-23-welcome-to-the-foo.zone.html | 8 +- ...022-02-04-computer-operating-systems-i-use.html | 42 +- gemfeed/2022-03-06-the-release-of-dtail-4.0.0.html | 28 +- gemfeed/2022-04-10-creative-universe.html | 22 +- .../2022-05-27-perl-is-still-a-great-choice.html | 12 +- gemfeed/2022-06-15-sweating-the-small-stuff.html | 44 +- ...22-07-30-lets-encrypt-with-openbsd-and-rex.html | 26 +- ...2-08-27-gemtexter-1.1.0-lets-gemtext-again.html | 20 +- gemfeed/2022-09-30-after-a-bad-nights-sleep.html | 32 +- .../2022-10-30-installing-dtail-on-openbsd.html | 22 +- ...-tried-emacs-but-i-switched-back-to-neovim.html | 16 +- ...22-12-24-ultrarelearning-java-my-takeaways.html | 24 +- gemfeed/2023-01-23-why-grapheneos-rox.html | 18 +- .../2023-02-26-how-to-shut-down-after-work.html | 16 +- ...-03-16-the-pragmatic-programmer-book-notes.html | 8 +- ...03-25-gemtexter-2.0.0-lets-gemtext-again-2.html | 14 +- ...4-01-never-split-the-difference-book-notes.html | 24 +- ...ithms-and-data-structures-in-golang-part-1.html | 12 +- ...iling-guprecords:-uptime-records-with-raku.html | 10 +- ...3-05-06-the-obstacle-is-the-way-book-notes.html | 26 +- ...3-06-01-kiss-server-monitoring-with-gogios.html | 30 +- ...17-career-guide-and-soft-skills-book-notes.html | 58 +-- ...07-21-gemtexter-2.1.0-lets-gemtext-again-3.html | 16 +- ...-08-18-site-reliability-engineering-part-1.html | 4 +- gemfeed/2023-09-25-dtail-usage-examples.html | 20 +- ...static-web-photo-albums-with-photoalbum.sh.html | 18 +- gemfeed/2023-11-11-mind-management-book-notes.html | 18 +- ...-11-19-site-reliability-engineering-part-2.html | 4 +- gemfeed/2023-12-10-bash-golf-part-3.html | 18 +- ...-01-09-site-reliability-engineering-part-3.html | 4 +- .../2024-01-13-one-reason-why-i-love-openbsd.html | 2 +- ...4-02-04-from-babylon5.buetow.org-to-.cloud.html | 28 +- ...quickly-logging-ideas-programmed-in-golang.html | 4 +- ...-04-01-KISS-high-availability-with-OpenBSD.html | 48 +- .../2024-05-01-slow-productivity-book-notes.html | 12 +- .../DRAFT-KISS-high-availability-with-OpenBSD.html | 22 +- gemfeed/DRAFT-site-reliability-engineering.html | 18 +- gemfeed/atom.xml | 522 +++++++++++---------- gemfeed/index.html | 4 +- index.html | 12 +- notes/a-monks-guide-to-happiness.html | 16 +- notes/career-guide-and-soft-skills.html | 58 +-- notes/index.html | 4 +- notes/influence-wihout-authority.html | 14 +- notes/mental-combat.html | 8 +- notes/mind-management.html | 18 +- notes/never-split-the-difference.html | 24 +- notes/slow-productivity.html | 12 +- notes/staff-engineer.html | 8 +- notes/the-obstacle-is-the-way.html | 26 +- notes/the-pragmatic-programmer.html | 8 +- notes/the-stoic-challenge.html | 6 +- other-resources.html | 2 +- resources.html | 2 +- site-mirrors.html | 8 +- testpage.html | 8 +- uptime-stats.html | 26 +- 82 files changed, 1001 insertions(+), 997 deletions(-) diff --git a/gemfeed/2008-06-26-perl-poetry.html b/gemfeed/2008-06-26-perl-poetry.html index 055e75b9..7d1ce809 100644 --- a/gemfeed/2008-06-26-perl-poetry.html +++ b/gemfeed/2008-06-26-perl-poetry.html @@ -8,7 +8,7 @@ -

Perl Poetry


+

Perl Poetry



Published at 2008-06-26T21:43:51+01:00; Updated at 2021-05-04

@@ -41,7 +41,7 @@ _~~|~/_|_|__/|~~~~~~~ | / ~~~~~ | | ~~~~~~~~
https://en.wikipedia.org/wiki/Perl

-

math.pl


+

math.pl



deriving Show
-

Processing a multi


+

Processing a multi



Standard ML:

@@ -74,7 +74,7 @@ test_number w = number ((Union (Union (Elem 4) (Elem 4)) Empty)))) w
-

Simplify function


+

Simplify function



Standard ML:

@@ -116,7 +116,7 @@ http://www.gnu.org/software/src-highlite --> simplify x = x
-

Delete all


+

Delete all



Standard ML:

@@ -145,7 +145,7 @@ http://www.gnu.org/software/src-highlite --> delete_all’ x = x
-

Delete one


+

Delete one



Standard ML:

@@ -191,7 +191,7 @@ http://www.gnu.org/software/src-highlite --> delete_one’ x = (x, False)
-

Higher-order functions


+

Higher-order functions



The first line is always the SML code, the second line the Haskell variant:

diff --git a/gemfeed/2010-05-07-lazy-evaluation-with-standarn-ml.html b/gemfeed/2010-05-07-lazy-evaluation-with-standarn-ml.html index ce4ba30a..7dcabdaf 100644 --- a/gemfeed/2010-05-07-lazy-evaluation-with-standarn-ml.html +++ b/gemfeed/2010-05-07-lazy-evaluation-with-standarn-ml.html @@ -8,7 +8,7 @@ -

Lazy Evaluation with Standard ML


+

Lazy Evaluation with Standard ML



Published at 2010-05-07T08:17:59+01:00

@@ -32,7 +32,7 @@
You can solve specific problems with lazy evaluation easier than with eager evaluation. For example, you might want to list the number Pi or another infinite list of something. With the help of lazy evaluation, each element of the list is calculated when it is accessed first, but not earlier.

-

Emulating lazy evaluation in SML


+

Emulating lazy evaluation in SML



However, it is possible to emulate lazy evaluation in most eager evaluation languages. This is how it is done with Standard ML (with some play with an infinite list of natural number tuples filtering out 0 elements):

@@ -78,7 +78,7 @@ val test = first 10 (nat_pairs_not_null ());
http://smlnj.org/

-

Real laziness with Haskell


+

Real laziness with Haskell



As Haskell already uses lazy evaluation by default, there is no need to construct a new data type. Lists in Haskell are lazy by default. You will notice that the code is also much shorter and easier to understand than the SML version.

diff --git a/gemfeed/2010-05-09-the-fype-programming-language.html b/gemfeed/2010-05-09-the-fype-programming-language.html index 454a4621..1728107d 100644 --- a/gemfeed/2010-05-09-the-fype-programming-language.html +++ b/gemfeed/2010-05-09-the-fype-programming-language.html @@ -8,7 +8,7 @@ -

The Fype Programming Language


+

The Fype Programming Language



Published at 2010-05-09T12:48:29+01:00; Updated at 2021-05-05

@@ -27,7 +27,7 @@
Fype is a recursive acronym and means "Fype is For Your Program Execution" or "Fype is Free Yak Programmed for ELF". You could also say, "It's not a hype - it's Fype!".

-

Object-oriented C style


+

Object-oriented C style



The Fype interpreter is written in an object-oriented style of C. Each "main component" has its own .h and .c file. There is a struct type for each (most components at least) component, which can be initialized using a "COMPONENT_new" function and destroyed using a "COMPONENT_delete" function. Method calls follow the same schema, e.g. "COMPONENT_METHODNAME". There is no such as class inheritance and polymorphism involved.

@@ -100,7 +100,7 @@ http://www.gnu.org/software/src-highlite --> }
-

Data types


+

Data types



Fype uses auto type conversion. However, if you want to know what's going on, you may take a look at the following basic data types:

There is no boolean type, but we can use the integer values 0 for false and 1 for true. There is support for explicit type casting too.

-

Syntax


+

Syntax



-

Comments


+

Comments



Text from a # character until the end of the current line is considered being a comment. Multi-line comments may start with an #* and with a *# anywhere. Exceptions are if those signs are inside of strings.

-

Variables


+

Variables



Variables are defined with the "my" keyword (inspired by Perl :-). If you don't assign a value during declaration, it uses the default integer value 0. Variables may be changed during program runtime. Variables may be deleted using the "undef" keyword! Example:

@@ -151,7 +151,7 @@ if defined foo { }
-

Synonyms


+

Synonyms



Each variable can have as many synonyms as wished. A synonym is another name to access the content of a specific variable. Here is an example of how to use it:

@@ -193,7 +193,7 @@ undef baz; say syms foo; # Prints 1
-

Statements and expressions


+

Statements and expressions



A Fype program is a list of statements. Each keyword, expression or function call is part of a statement. Each statement is ended with a semicolon. Example:

@@ -203,11 +203,11 @@ say foo; exit foo - bar;
-

Parenthesis


+

Parenthesis



All parenthesis for function arguments is optional. They help to make the code better readable. They also help to force the precedence of expressions.

-

Basic expressions


+

Basic expressions



Any "any" value holding a string will be automatically converted to an integer value.

@@ -225,7 +225,7 @@ exit foo - bar; (integer) not <any>
-

Bitwise expressions


+

Bitwise expressions



 (integer) <any> :< <any>
@@ -235,7 +235,7 @@ exit foo - bar;
 (integer) <any> xor <any>

-

Numeric expressions


+

Numeric expressions



 (number) neg <number>
@@ -260,7 +260,7 @@ exit foo - bar;
 if yes { say no defined foo; }

-

Control statements


+

Control statements



Control statements available in Fype:

@@ -288,7 +288,7 @@ until <expression> { <statements> }
... runs the statements as long as the expression evaluates to a false value.

-

Scopes


+

Scopes



A new scope starts with an { and ends with an }. An exception is a procedure, which does not use its own scope (see later in this manual). Control statements and functions support scopes. The "scope" function prints out all available symbols at the current scope. Here is a small example:

@@ -339,7 +339,7 @@ SYM_VARIABLE: var4 (id=00035, line=-0001, pos=-001, type=TT_INTEGER, dval=0.0000 SYM_FUNCTION: bar
-

Definedness


+

Definedness 



 (integer) defined <identifier>
@@ -353,7 +353,7 @@ SYM_FUNCTION: bar
 

 ... tries to undefine/delete the "identifier". Returns 1 if it succeeded, otherwise 0 is returned.

 

-
System 


+
System 


 

 These are some system and interpreter specific built-in functions supported:

 

@@ -395,7 +395,7 @@ if pid {
 

 It returns the number of items freed! You may wonder why most of the time, it will produce a value of 0! Fype tries to free not needed memory ASAP. This may change in future versions to gain faster execution speed!

 

-
I/O 


+
I/O 


 

 
 (any) put <any>
@@ -415,9 +415,9 @@ if pid {
 

 ... just prints a new line.

 

-
Procedures and functions


+
Procedures and functions


 

-
Procedures


+
Procedures


 

 A procedure can be defined with the "proc" keyword and deleted with the "undef" keyword. A procedure does not return any value and does not support parameter passing. It's using already defined variables (e.g. global variables). A procedure does not have its own namespace. It's using the calling namespace. It is possible to define new variables inside of a procedure in the current namespace.

 

@@ -433,7 +433,7 @@ foo; # Run the procedure. Print out "11\n"
 say c; # Print out "6\n";
 

 

-
Nested procedures


+
Nested procedures


 

 It's possible to define procedures inside of procedures. Since procedures don't have their own scope, nested procedures will be available to the current scope as soon as the main procedure has run the first time. You may use the "defined" keyword to check if a procedure has been defined or not.

 

@@ -456,7 +456,7 @@ bar; # Now the procedure bar is defined!
 foo; # Here the procedure foo will redefine bar again!

-

Functions


+

Functions



A function can be defined with the "func" keyword and deleted with the "undef" keyword. Function do not yet return values and do not yet supports parameter passing. It's using local (lexical scoped) variables. If a certain variable does not exist, when It's using already defined variables (e.g. one scope above).

@@ -472,7 +472,7 @@ foo; # Run the procedure. Print out "11\n" say c; # Will produce an error because c is out of scope!
-

Nested functions


+

Nested functions



Nested functions work the same way the nested procedures work, except that nested functions will not be available anymore after the function has been left!

@@ -489,7 +489,7 @@ foo; bar; # Will produce an error because bar is out of scope!
-

Arrays


+

Arrays



Some progress on arrays has been made too. The following example creates a multidimensional array "foo". Its first element is the return value of the func which is "bar". The fourth value is a string" 3" converted to a double number. The last element is an anonymous array which itself contains another anonymous array as its final element:

@@ -512,11 +512,11 @@ BA BB
-

Fancy stuff


+

Fancy stuff



Fancy stuff like OOP or Unicode or threading is not planed. But fancy stuff like function pointers and closures may be considered.:)

-

May the source be with you


+

May the source be with you



You can find all of this on the GitHub page. There is also an "examples" folders containing some Fype scripts!

diff --git a/gemfeed/2011-05-07-perl-daemon-service-framework.html b/gemfeed/2011-05-07-perl-daemon-service-framework.html index 66e2690a..34ee06d9 100644 --- a/gemfeed/2011-05-07-perl-daemon-service-framework.html +++ b/gemfeed/2011-05-07-perl-daemon-service-framework.html @@ -8,7 +8,7 @@ -

Perl Daemon (Service Framework)


+

Perl Daemon (Service Framework)



Published at 2011-05-07T22:26:02+01:00; Updated at 2021-05-07

@@ -21,7 +21,7 @@
PerlDaemon is a minimal daemon for Linux and other Unix like operating systems programmed in Perl. It is a minimal but pretty functional and fairly generic service framework. This means that it does not do anything useful other than providing a framework for starting, stopping, configuring and logging. To do something useful, a module (written in Perl) must be provided.

-

Features


+

Features



PerlDaemon supports:

@@ -35,7 +35,7 @@
  • Easy to extend
  • Multi-instance support (just use a different directory for each instance).

    • -

    Quick Guide


    +

    Quick Guide




    To stop a daemon from running in foreground mode, "Ctrl+C" must be hit. To see more available startup options run "./control" without any argument.

    -

    How to configure


    +

    How to configure



    The daemon instance can be configured in "./conf/perldaemon.conf". If you want to change a property only once, it is also possible to specify it on the command line (which will take precedence over the config file). All available config properties can be displayed via "./control keys":

    @@ -87,7 +87,7 @@ daemon.alivefile.wd=./
    -

    Example


    +

    Example



    So let's start the daemon with a loop interval of 10 seconds:

    @@ -117,13 +117,13 @@ http://www.gnu.org/software/src-highlite --> 
    $ ./control keys daemon.loopinterval=10 > new.conf; mv new.conf conf/perldaemon.conf

    -

    HiRes event loop


    +

    HiRes event loop



    PerlDaemon uses Time::HiRes to make sure that all the events run incorrect intervals. For each loop run, a time carry value is recorded and added to the next loop run to catch up on lost time.

    -

    Writing your own modules


    +

    Writing your own modules



    -

    Example module


    +

    Example module



    This is one of the example modules you will find in the source code. It should be pretty self-explanatory if you know Perl :-).

    @@ -162,7 +162,7 @@ http://www.gnu.org/software/src-highlite --> 1;
    -

    Your own module


    +

    Your own module



    Want to give it some better use? It's just as easy as:

    @@ -181,7 +181,7 @@ http://www.gnu.org/software/src-highlite -->
    BTW: You can install as many modules within the same instance as desired. But they are run in sequential order (in future, they can also run in parallel using several threads or processes).

    -

    May the source be with you


    +

    May the source be with you



    You can find PerlDaemon (including the examples) at:

    diff --git a/gemfeed/2014-03-24-the-fibonacci.pl.c-polyglot.html b/gemfeed/2014-03-24-the-fibonacci.pl.c-polyglot.html index 06f3a867..35725876 100644 --- a/gemfeed/2014-03-24-the-fibonacci.pl.c-polyglot.html +++ b/gemfeed/2014-03-24-the-fibonacci.pl.c-polyglot.html @@ -8,7 +8,7 @@ -

    The fibonacci.pl.raku.c Polyglot


    +

    The fibonacci.pl.raku.c Polyglot



    Published at 2014-03-24T21:32:53+00:00; Updated at 2022-04-23

    @@ -16,7 +16,7 @@
    https://en.wikipedia.org/wiki/Polyglot_(computing)

    -

    The Fibonacci numbers


    +

    The Fibonacci numbers



    For fun, I programmed my own Polyglot, which is both valid Perl, Raku, C and C++ code (I have added C++ and Raku support in 2022). The exciting part about C and C++ is that $ is a valid character to start variable names with:

    @@ -71,7 +71,7 @@ BEGIN {
    https://codeberg.org/snonux/perl-c-fibonacci

    -

    Let's run it with C and C++


    +

    Let's run it with C and C++



    exit # Leave adb shell
    -

    Setup of various scripts


    +

    Setup of various scripts



    jessie.sh deals with all the loopback mount magic and so on. It will be run later every time you start Debroid on your phone.

    @@ -164,7 +164,7 @@ apt-get dist-upgrade exit # Exit chroot
    -

    Entering Debroid and enable a service


    +

    Entering Debroid and enable a service



    This enters Debroid on your phone and starts the example service uptimed:

    @@ -187,7 +187,7 @@ chmod 0755 /etc/rc.deb exit # Exit adb shell
    -

    Include to Android startup:


    +

    Include to Android startup:



    If you want to start Debroid automatically whenever your phone starts, then do the following:

    diff --git a/gemfeed/2016-04-03-offsite-backup-with-zfs.html b/gemfeed/2016-04-03-offsite-backup-with-zfs.html index 5c3043b1..c7a5bee8 100644 --- a/gemfeed/2016-04-03-offsite-backup-with-zfs.html +++ b/gemfeed/2016-04-03-offsite-backup-with-zfs.html @@ -8,7 +8,7 @@ -

    Offsite backup with ZFS


    +

    Offsite backup with ZFS



    Published at 2016-04-03T22:43:42+01:00

    @@ -28,25 +28,25 @@ Offsite backup with ZFS Part 1 (you are reading this atm.)
    Offsite backup with ZFS Part 2

    -

    Please don't lose all my pictures again!


    +

    Please don't lose all my pictures again!



    When it comes to data storage and potential data loss, I am a paranoid person. It is due to my job and a personal experience I encountered over ten years ago: A single drive failure and loss of all my data (pictures, music, etc.).

    A little about my personal infrastructure: I am running my own (mostly FreeBSD based) root servers (across several countries: Two in Germany, one in Canada, one in Bulgaria) which store all my online data (E-Mail and my Git repositories). I am syncing incremental (and encrypted) ZFS snapshots between these servers forth and back so either data can be recovered from the other server.

    -

    Local storage box for offline data


    +

    Local storage box for offline data



    Also, I am operating a local server (an HP MicroServer) at home in my apartment. Full snapshots of all ZFS volumes are pulled from the "online" servers to the local server every other week and the incremental ZFS snapshots every day. That local server has a ZFS ZMIRROR with three disks configured (local triple redundancy). I keep up to half a year worth of ZFS snapshots of all volumes. That local server also contains all my offline data such as pictures, private documents, videos, books, various other backups, etc.

    Once weekly, all the local server data is copied to two external USB drives as a backup (without the historic snapshots). For simplicity, these USB drives are not formatted with ZFS but with good old UFS. This gives me a chance to recover from a (potential) ZFS disaster. ZFS is a complex thing. Sometimes it is good not to trust complicated things!

    -

    Storing it at my apartment is not enough


    +

    Storing it at my apartment is not enough



    Now I am thinking about an offsite backup of all this local data. The problem is that all the data remains on a single physical location: My local MicroServer. What happens when the house burns or my server, including the internal disks and the attached USB drives, gets stolen? My first thought was to back up everything to the "cloud". However, the significant issue here is the limited amount of available upload bandwidth (only 1MBit/s).

    The solution is adding another USB drive (2TB) with an encryption container (GELI) and a ZFS pool. The GELI encryption requires a secret key and a secret passphrase. I am updating the data to that drive once every three months (my calendar is reminding me about it), and afterwards, I keep that drive at a secret location outside of my apartment. All the information needed to decrypt (mounting the GELI container) is stored at another (secure) place. Key and passphrase are kept at different sites, though. Even if someone knew of it, he would not be able to decrypt it as some additional insider knowledge would be required as well.

    -

    Walking one round less


    +

    Walking one round less



    I am thinking of buying a second 2TB USB drive and setting it up the same way as the first one. So I could alternate the backups. One drive would be at the secret location, and the other drive would be at home. And these drives would swap place after each cycle. This would give some security about the failure of that drive, and I would have to go to the secret location only once (swapping the drives) instead of twice (picking that drive up to update the data + bringing it back to the remote location).

    diff --git a/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.html b/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.html index ac6c664f..3393e028 100644 --- a/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.html +++ b/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.html @@ -8,7 +8,7 @@ -

    Jails and ZFS with Puppet on FreeBSD


    +

    Jails and ZFS with Puppet on FreeBSD



    Published at 2016-04-09T18:29:47+01:00

    @@ -34,7 +34,7 @@
    Over the last couple of years I wrote quite a few Puppet modules in order to manage my personal server infrastructure. One of them manages FreeBSD Jails and another one ZFS file systems. I thought I would give a brief overview in how it looks and feels.

    -

    ZFS


    +

    ZFS



    The ZFS module is a pretty basic one. It does not manage ZFS pools yet as I am not creating them often enough which would justify implementing an automation. But let's see how we can create a ZFS file system (on an already given ZFS pool named ztank):

    @@ -105,7 +105,7 @@ zsh: done df | zsh: exit 1 grep foo
    -

    Jails


    +

    Jails



    Here is an example in how a FreeBSD Jail can be created. The Jail will have its own public IPv6 address. And it will have its own internal IPv4 address with IPv4 NAT to the internet (this is due to the limitation that the host server only got one public IPv4 address which requires sharing between all the Jails).

    @@ -242,7 +242,7 @@ lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
    -

    Inside-Jail Puppet


    +

    Inside-Jail Puppet



    To automatically setup the applications running in the Jail I am using Puppet as well. I wrote a few scripts which bootstrap Puppet inside of a newly created Jail. It is doing the following:

    @@ -383,7 +383,7 @@ Notice: /Stage[main]/S_user::Root_files/S_user::My_files[root]/File[/root/.task] Notice: Finished catalog run in 206.09 seconds
    -

    Managing multiple Jails


    +

    Managing multiple Jails



    Of course I am operating multiple Jails on the same host this way with Puppet:

    diff --git a/gemfeed/2016-04-16-offsite-backup-with-zfs-part2.html b/gemfeed/2016-04-16-offsite-backup-with-zfs-part2.html index e9aebb26..ce199ab8 100644 --- a/gemfeed/2016-04-16-offsite-backup-with-zfs-part2.html +++ b/gemfeed/2016-04-16-offsite-backup-with-zfs-part2.html @@ -8,7 +8,7 @@ -

    Offsite backup with ZFS (Part 2)


    +

    Offsite backup with ZFS (Part 2)



    Published at 2016-04-16T22:43:42+01:00

    diff --git a/gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.html b/gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.html index a4cd7559..9dcafdfc 100644 --- a/gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.html +++ b/gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.html @@ -8,17 +8,17 @@ -

    Spinning up my own authoritative DNS servers


    +

    Spinning up my own authoritative DNS servers



    Published at 2016-05-22T18:59:01+01:00

    -

    Background


    +

    Background



    Finally, I had time to deploy my authoritative DNS servers (master and slave) for my domains "buetow.org" and "buetow.zone". My domain name provider is Schlund Technologies. They allow their customers to edit the DNS records (BIND files) manually. And they also allow you to set your authoritative DNS servers for your domains. From now, I am making use of that option.

    Schlund Technologies

    -

    All FreeBSD Jails


    +

    All FreeBSD Jails



    To set up my authoritative DNS servers, I installed a FreeBSD Jail dedicated for DNS with Puppet on my root machine as follows:

    @@ -55,7 +55,7 @@ class { 'jail': }
    -

    PF firewall


    +

    PF firewall



    Please note that "dns.ian.buetow.org" is just the Jail name of the master DNS server (and "caprica.ian.buetow.org" the name of the Jail for the slave DNS server) and that I am using the DNS names "dns1.buetow.org" (master) and "dns2.buetow.org" (slave) for the actual service names (these are the DNS servers visible to the public). Please also note that the IPv4 address is an internal one. I have a PF to use NAT and PAT. The DNS ports are being forwarded (TCP and UDP) to that Jail. By default, all ports are blocked, so I am adding an exception rule for the IPv6 address. These are the PF rules in use:

    @@ -72,7 +72,7 @@ pass in on re0 inet6 proto udp from any to 2a01:4f8:120:30e8::15 port {53} flags .
    -

    Puppet managed BIND zone files


    +

    Puppet managed BIND zone files



    In "manifests/dns.pp" (the Puppet manifest for the Master DNS Jail itself), I configured the BIND DNS server this way:

    @@ -146,7 +146,7 @@ zone "buetow.zone" { };
    -

    The result


    +

    The result



    The result looks like this now:

    @@ -206,7 +206,7 @@ dns2.buetow.org. 86400 IN AAAA 2a03:2500:1:6:20:: ;; MSG SIZE rcvd: 322
    -

    Monitoring


    +

    Monitoring



    For monitoring, I am using Icinga2 (I am operating two Icinga2 instances in two different DCs). I may have to post another blog article about Icinga2, but to get the idea, these were the snippets added to my Icinga2 configuration:

    @@ -233,7 +233,7 @@ apply Service "dig6" { }
    -

    DNS update workflow


    +

    DNS update workflow



    Whenever I have to change a DNS entry, all I have to do is:

    diff --git a/gemfeed/2016-11-20-object-oriented-programming-with-ansi-c.html b/gemfeed/2016-11-20-object-oriented-programming-with-ansi-c.html index 05ec8102..3c8e6fad 100644 --- a/gemfeed/2016-11-20-object-oriented-programming-with-ansi-c.html +++ b/gemfeed/2016-11-20-object-oriented-programming-with-ansi-c.html @@ -8,7 +8,7 @@ -

    Object oriented programming with ANSI C


    +

    Object oriented programming with ANSI C



    Published at 2016-11-20T22:10:57+00:00; Updated at 2022-01-29

    @@ -23,7 +23,7 @@
    You can do a little of object-oriented programming in the C Programming Language. However, that is, in my humble opinion, limited. It's easier to use a different programming language than C for OOP. But still it's an interesting exercise to try using C for this.

    -

    Function pointers


    +

    Function pointers



    Let's have a look at the following sample program. All you have to do is to add a function pointer such as "calculate" to the definition of struct "something_s". Later, during the struct initialization, assign a function address to that function pointer:

    @@ -98,7 +98,7 @@ Division(3.000000 Not complicated at all, but nice to know and helps to make the code easier to read!

    -

    That's not OOP, though


    +

    That's not OOP, though



    However, that's not really how it works in object-oriented languages such as Java and C++. The method call in this example is not a method call as "mult" and "div" in this example are not "message receivers". I mean that the functions can not access the state of the "mult" and "div" struct objects. In C, you would need to do something like this instead if you wanted to access the state of "mult" from within the calculate function, you would have to pass it as an argument:

    @@ -109,13 +109,13 @@ http://www.gnu.org/software/src-highlite --> 
    mult.calculate(mult,a,b));

    -

    Real object oriented programming with C


    +

    Real object oriented programming with C



    If you want to take it further, hit "Object-Oriented Programming with ANSI-C" into your favourite internet search engine or follow the link below. It goes as far as writing a C preprocessor in AWK, which takes some object-oriented pseudo-C and transforms it to plain C so that the C compiler can compile it to machine code. This is similar to how the C++ language had its origins.

    https://www.cs.rit.edu/~ats/books/ooc.pdf

    -

    OOP design patterns in the Linux Kernel


    +

    OOP design patterns in the Linux Kernel



    Big C software projects, like Linux, also follow some OOP techniques:

    diff --git a/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.html b/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.html index 80bc585e..b544bcbd 100644 --- a/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.html +++ b/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.html @@ -8,7 +8,7 @@ -

    Realistic load testing with I/O Riot for Linux


    +

    Realistic load testing with I/O Riot for Linux



    Published at 2018-06-01T14:50:29+01:00; Updated at 2021-05-08

    @@ -23,7 +23,7 @@ jgs\__/'---'\__/
    -

    Foreword


    +

    Foreword



    This text first was published in the german IT-Administrator computer Magazine. 3 years have passed since and I decided to publish it on my blog too.

    @@ -35,13 +35,13 @@ jgs\__/'---'\__/
    Pleace notice that some of the screenshots show the command "ioreplay" instead of "ioriot". That's because the name has changed after taking those.

    -

    The article


    +

    The article



    With I/O Riot IT administrators can load test and optimize the I/O subsystem of Linux-based operating systems. The tool makes it possible to record I/O patterns and replay them at a later time as often as desired. This means bottlenecks can be reproduced and eradicated.

    When storing huge amounts of data, such as more than 200 billion archived emails at Mimecast, it's not only the available storage capacity that matters, but also the data throughput and latency. At the same time, operating costs must be kept as low as possible. The more systems involved, the more important it is to optimize the hardware, the operating system and the applications running on it.

    -

    Background: Existing Techniques


    +

    Background: Existing Techniques



    Conventional I/O benchmarking: Administrators usually use open source benchmarking tools like IOZone and bonnie++. Available database systems such as Redis and MySQL come with their own benchmarking tools. The common problem with these tools is that they work with prescribed artificial I/O patterns. Although this can test both sequential and randomized data access, the patterns do not correspond to what can be found on production systems.

    @@ -49,17 +49,17 @@ jgs\__/'---'\__/
    Testing in the production environment: For these reasons, benchmarks are often carried out in the production environment. In order to derive value from this such tests are especially performed during peak hours when systems are under high load. However, testing on production systems is associated with risks and can lead to failure or loss of data without adequate protection.

    -

    Benchmarking the Email Cloud at Mimecast


    +

    Benchmarking the Email Cloud at Mimecast



    For email archiving, Mimecast uses an internally developed microservice, which is operated directly on Linux-based storage systems. A storage cluster is divided into several replication volumes. Data is always replicated three times across two secure data centers. Customer data is automatically allocated to one or more volumes, depending on throughput, so that all volumes are automatically assigned the same load. Customer data is archived on conventional, but inexpensive hard disks with several terabytes of storage capacity each. I/O benchmarking proved difficult for all the reasons mentioned above. Furthermore, there are no ready-made tools for this purpose in the case of self-developed software. The service operates on many block devices simultaneously, which can make the RAID controller a bottleneck. None of the freely available benchmarking tools can test several block devices at the same time without extra effort. In addition, emails typically consist of many small files. Randomized access to many small files is particularly inefficient. In addition to many software adaptations, the hardware and operating system must also be optimized.

    Mimecast encourages employees to be innovative and pursue their own ideas in the form of an internal competition, Pet Project. The goal of the pet project I/O Riot was to simplify OS and hardware level I/O benchmarking. The first prototype of I/O Riot was awarded an internal roadmap prize in the spring of 2017. A few months later, I/O Riot was used to reduce write latency in the storage clusters by about 50%. The improvement was first verified by I/O replay on a test system and then successively applied to all storage systems. I/O Riot was also used to resolve a production incident caused by disk I/O load.

    -

    Using I/O Riot


    +

    Using I/O Riot



    First, all I/O events are logged to a file on a production system with I/O Riot. It is then copied to a test system where all events are replayed in the same way. The crucial point here is that you can reproduce I/O patterns as they are found on a production system as often as you like on a test system. This results in the possibility of optimizing the set screws on the system after each run.

    -

    Installation


    +

    Installation



    I/O Riot was tested under CentOS 7.2 x86_64. For compiling, the GNU C compiler and Systemtap including kernel debug information are required. Other Linux distributions are theoretically compatible but untested. First of all, you should update the systems involved as follows:

    @@ -81,7 +81,7 @@ jgs\__/'---'\__/
    Note: It is not best practice to install any compilers on production systems. For further information please have a look at the enclosed README.md.

    -

    Recording of I/O events


    +

    Recording of I/O events



    All I/O events are kernel related. If a process wants to perform an I/O operation, such as opening a file, it must inform the kernel of this by a system call (short syscall). I/O Riot relies on the Systemtap tool to record I/O syscalls. Systemtap, available for all popular Linux distributions, helps you to take a look at the running kernel in productive environments, which makes it predestined to monitor all I/O-relevant Linux syscalls and log them to a file. Other tools, such as strace, are not an alternative because they slow down the system too much.

    @@ -95,7 +95,7 @@ jgs\__/'---'\__/
    A Ctrl-C (SIGINT) stops recording prematurely. Otherwise, ioriot terminates itself automatically after 1 hour. Depending on the system load, the output file can grow to several gigabytes. Only metadata is logged, not the read and written data itself. When replaying later, only random data is used. Under certain circumstances, Systemtap may omit some system calls and issue warnings. This is to ensure that Systemtap does not consume too many resources.

    -

    Test preparation


    +

    Test preparation



    Then copy io.capture to a test system. The log also contains all accesses to the pseudo file systems devfs, sysfs and procfs. This makes little sense, which is why you must first generate a cleaned and playable version io.replay from io.capture as follows:

    @@ -105,7 +105,7 @@ jgs\__/'---'\__/
    The parameter -n allows you to assign a freely selectable test name. An arbitrary system user under which the test is to be played is specified via paramater -u.

    -

    Test Initialization


    +

    Test Initialization



    The test will most likely want to access existing files. These are files the test wants to read but does not create by itself. The existence of these must be ensured before the test. You can do this as follows:

    @@ -119,7 +119,7 @@ jgs\__/'---'\__/
    You must re-initialize the environment before each run. Data from previous tests will be moved to a trash directory automatically, which can be finally deleted with "sudo ioriot -P".

    -

    Replay


    +

    Replay



    After initialization, you can replay the log with -r. You can use -R to initiate both test initialization and replay in a single command and -S can be used to specify a file in which statistics are written after the test run.

    @@ -175,13 +175,13 @@ Total time: 1213.00s
    Output of iostat. The block device sdy seems to be almost fully utilized by 99%.

    -

    I/O Riot is Open Source


    +

    I/O Riot is Open Source



    The tool has already proven to be very useful and will continue to be actively developed as time and priority permits. Mimecast intends to be an ongoing contributor to Open Source. You can find I/O Riot at:

    https://github.com/mimecast/ioriot

    -

    Systemtap


    +

    Systemtap



    Systemtap is a tool for the instrumentation of the Linux kernel. The tool provides an AWK-like programming language. Programs written in it are compiled from Systemtap to C- and then into a dynamically loadable kernel module. Loaded into the kernel, the program has access to Linux internals. A Systemtap program written for I/O Riot monitors when, with which parameters, at which time, and from which process I/O syscalls take place and their return values.

    @@ -189,7 +189,7 @@ Total time: 1213.00s
    https://sourceware.org/systemtap/

    -

    More refereces


    +

    More refereces



    IOZone
    Bonnie++
    diff --git a/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.html b/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.html index 68fcfcf7..48b450c0 100644 --- a/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.html +++ b/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.html @@ -8,7 +8,7 @@ -

    DTail - The distributed log tail program


    +

    DTail - The distributed log tail program



    Published at 2021-04-22T19:28:41+01:00; Updated at 2021-04-26

    @@ -26,7 +26,7 @@
    Think of DTail as a distributed version of the tail program which is very useful when you have a distributed application running on many servers. DTail is an Open-Source, cross-platform, fairly easy to use, support and maintain log file analysis & statistics gathering tool designed for Engineers and Systems Administrators. It is programmed in Google Go.

    -

    A Mimecast Pet Project


    +

    A Mimecast Pet Project



    DTail got its inspiration from public domain tools available already in this area but it is a blue sky from-scratch development which was first presented at Mimecast’s annual internal Pet Project competition (awarded with a Bronze prize). It has gained popularity since and is one of the most widely deployed DevOps tools at Mimecast (reaching nearly 10k server installations) and many engineers use it on a regular basis. The Open-Source version of DTail is available at: