diff options
Diffstat (limited to 'content')
21 files changed, 3467 insertions, 80 deletions
diff --git a/content/gemtext/gemfeed/2008-06-26-perl-poetry.gmi b/content/gemtext/gemfeed/2008-06-26-perl-poetry.gmi new file mode 100644 index 00000000..1ba53967 --- /dev/null +++ b/content/gemtext/gemfeed/2008-06-26-perl-poetry.gmi @@ -0,0 +1,166 @@ +> Written by Paul Buetow 2008-06-26, last updated 2021-05-04 + +=> ../ Go back to the main site + +# Perl Poetry + +``` + '\|/' * +-- * ----- + /|\ ____ + ' | ' {_ o^> * + : -_ /) + : ( ( .-''`'. + . \ \ / \ + . \ \ / \ + \ `-' `'. + \ . ' / `. + \ ( \ ) ( .') + ,, t '. | / | ( + '|``_/^\___ '| |`'-..-'| ( () +_~~|~/_|_|__/|~~~~~~~ | / ~~~~~ | | ~~~~~~~~ + -_ |L[|]L|/ | |\ MJP ) ) + ( |( / /| + ~~ ~ ~ ~~~~ | /\\ / /| | + || \\ _/ / | | + ~ ~ ~~~ _|| (_/ (___)_| |Nov291999 + (__) (____) +``` + +Here are some Perl Poems I wrote. They don't do anything useful when you run them, but they don't produce a compiler error either. They only exist for fun and demonstrate what you can do with Perl syntax. + +Wikipedia: "Perl poetry is the practice of writing poems that can be compiled as legal Perl code, for example the piece known as Black Perl. Perl poetry is made possible by the large number of English words that are used in the Perl language. New poems are regularly submitted to the community at PerlMonks." + +=> https://en.wikipedia.org/wiki/Perl + +## math.pl + +``` +#!/usr/bin/perl + +# (C) 2006 by Paul C. Buetow (http://paul.buetow.org) + +goto library for study $math; +BEGIN { s/earching/ books/ +and read $them, $at, $the } library: + +our $topics, cos and tan, +require strict; import { of, tied $patience }; + +do { int'egrate'; sub trade; }; +do { exp'onentize' and abs'olutize' }; +study and study and study and study; + +foreach $topic ({of, math}) { +you, m/ay /go, to, limits } + +do { not qw/erk / unless $success +and m/ove /o;$n and study }; + +do { int'egrate'; sub trade; }; +do { exp'onentize' and abs'olutize' }; +study and study and study and study; + +grep /all/, exp'onents' and cos'inuses'; +/seek results/ for @all, log'4rithms'; + +'you' =~ m/ay /go, not home +unless each %book ne#ars +$completion; + +do { int'egrate'; sub trade; }; +do { exp'onentize' and abs'olutize' }; + +#at +home: //ig,'nore', time and sleep $very =~ s/tr/on/g; +__END__ + +``` + +## christmas.pl + +``` +#!/usr/bin/perl + +# (C) 2006 by Paul C. Buetow (http://paul.buetow.org) + +Christmas:{time;#!!! + +Children: do tell $wishes; + +Santa: for $each (@children) { +BEGIN { read $each, $their, wishes and study them; use Memoize#ing + +} use constant gift, 'wrapping'; +package Gifts; pack $each, gift and bless $each and goto deliver +or do import if not local $available,!!! HO, HO, HO; + +redo Santa, pipe $gifts, to_childs; +redo Santa and do return if last one, is, delivered; + +deliver: gift and require diagnostics if our $gifts ,not break; +do{ use NEXT; time; tied $gifts} if broken and dump the, broken, ones; +The_children: sleep and wait for (each %gift) and try { to => untie $gifts }; + +redo Santa, pipe $gifts, to_childs; +redo Santa and do return if last one, is, delivered; + +The_christmas_tree: formline s/ /childrens/, $gifts; +alarm and warn if not exists $Christmas{ tree}, @t, $ENV{HOME}; +write <<EMail + to the parents to buy a new christmas tree!!!!111 + and send the +EMail +;wait and redo deliver until defined local $tree; + +redo Santa, pipe $gifts, to_childs; +redo Santa and do return if last one, is, delivered ;} + +END {} our $mission and do sleep until next Christmas ;} + +__END__ + +This is perl, v5.8.8 built for i386-freebsd-64int +``` + +## shopping.pl + +``` +#!/usr/bin/perl + +# (C) 2007 by Paul C. Buetow (http://paul.buetow.org) + +BEGIN{} goto mall for $shopping; + +m/y/; mall: seek$s, cool products(), { to => $sell }; +for $their (@business) { to:; earn:; a:; lot:; of:; money: } + +do not goto home and exit mall if exists $new{product}; +foreach $of (q(uality rich products)){} package products; + +our $news; do tell cool products() and do{ sub#tract +cool{ $products and shift @the, @bad, @ones; + +do bless [q(uality)], $products +and return not undef $stuff if not (local $available) }}; + +do { study and study and study for cool products() } +and do { seek $all, cool products(), { to => $buy } }; + +do { write $them, $down } and do { order: foreach (@case) { package s } }; +goto home if not exists $more{money} or die q(uerying) ;for( @money){}; + +at:;home: do { END{} and:; rest:; a:; bit: exit $shopping } +and sleep until unpack$ing, cool products(); + +__END__ +This is perl, v5.8.8 built for i386-freebsd-64int +``` + +## More... + +Did you like what you saw? Have a look at Github to see my other poems too: + +=> https://github.com/snonux/perl-poetry + +E-Mail me your thoughts at comments@mx.buetow.org! diff --git a/content/gemtext/gemfeed/2010-05-09-the-fype-programming-language.gmi b/content/gemtext/gemfeed/2010-05-09-the-fype-programming-language.gmi new file mode 100644 index 00000000..3b008457 --- /dev/null +++ b/content/gemtext/gemfeed/2010-05-09-the-fype-programming-language.gmi @@ -0,0 +1,511 @@ +> Written by Paul Buetow 2010-05-09, last updated 2021-05-05 + +=> ../ Go back to the main site + +``` + ____ _ __ + / / _|_ _ _ __ ___ _ _ ___ __ _| |__ / _|_ _ + / / |_| | | | '_ \ / _ \ | | | |/ _ \/ _` | '_ \ | |_| | | | + _ / /| _| |_| | |_) | __/ | |_| | __/ (_| | | | |_| _| |_| | +(_)_/ |_| \__, | .__/ \___| \__, |\___|\__,_|_| |_(_)_| \__, | + |___/|_| |___/ |___/ + +``` + +# The Fype Programming Language + +Fype is an interpreted programming language created by me for learning and fun. The interpreter is written in C. It has been tested on FreeBSD and NetBSD and may also work on other Unix like operating systems such as Linux based ones. To be honest, besides learning and fun there is really no other use case of why Fype actually exists as many other programming languages are much faster and more powerful. + +The Fype syntax is very simple and is using a maximum look ahead of 1 and a very easy top down parsing mechanism. Fype is parsing and interpreting its code simultaneously. This means, that syntax errors are only detected during program runtime. + +Fype is a recursive acronym and means "Fype is For Your Program Execution" or "Fype is Free Yak Programmed for ELF". You could also say "It's not a hype - it's Fype!". + +## Object oriented C style + +The Fype interpreter is written in an object oriented style of C. Each "main component" has its own .h and .c file. There is a struct type for each (most components at least) component which can be initialized using a "COMPONENT_new" function and destroyed using a "COMPONENT_delete" function. Method calls follow the same schema, e.g. "COMPONENT_METHODNAME". There is no such as class inheritance and polymorphism involved. + +To give you an idea how it works here as an example is a snippet from the main Fype "class header": + +``` +typedef struct { + Tupel *p_tupel_argv; // Contains command line options + List *p_list_token; // Initial list of token + Hash *p_hash_syms; // Symbol table + char *c_basename; +} Fype; +``` + +And here is a snippet from the main Fype "class implementation": + +``` +Fype* +fype_new() { + Fype *p_fype = malloc(sizeof(Fype)); + + p_fype->p_hash_syms = hash_new(512); + p_fype->p_list_token = list_new(); + p_fype->p_tupel_argv = tupel_new(); + p_fype->c_basename = NULL; + + garbage_init(); + + return (p_fype); +} + +void +fype_delete(Fype *p_fype) { + argv_tupel_delete(p_fype->p_tupel_argv); + + hash_iterate(p_fype->p_hash_syms, symbol_cleanup_hash_syms_cb); + hash_delete(p_fype->p_hash_syms); + + list_iterate(p_fype->p_list_token, token_ref_down_cb); + list_delete(p_fype->p_list_token); + + if (p_fype->c_basename) + free(p_fype->c_basename); + + garbage_destroy(); +} + +int +fype_run(int i_argc, char **pc_argv) { + Fype *p_fype = fype_new(); + + // argv: Maintains command line options + argv_run(p_fype, i_argc, pc_argv); + + // scanner: Creates a list of token + scanner_run(p_fype); + + // interpret: Interpret the list of token + interpret_run(p_fype); + + fype_delete(p_fype); + + return (0); +} +``` + +## Data types + +Fype uses auto type conversion. However, if you want to know what's going on you may take a look at the following basic data types: +* integer - Specifies a number +* double - Specifies a double precision number +* string - Specifies a string +* number - May be an integer or a double number +* any- May be any type above +* void - No type +* identifier - It's a variable name or a procedure name or a function name + +There is no boolean type, but we can use the integer values 0 for false and 1 for true. There is support for explicit type casting too. + +## Syntax + +### Comments + +Text from a # character until the end of the current line is considered being a comment. Multi line comments may start with an #* and with a *# anywhere. Exceptions are if those signs are inside of strings. + +### Variables + +Variables can be defined with the "my" keyword (inspired by Perl :-). If you don't assign a value during declaration, then it's using the default integer value 0. Variables may be changed during program runtime. Variables may be deleted using the "undef" keyword! Example: + +``` +my foo = 1 + 2; +say foo; + +my bar = 12, baz = foo; +say 1 + bar; +say bar; + +my baz; +say baz; # Will print out 0 +``` + +You may use the "defined" keyword to check if an identifier has been defined or not: + +``` +ifnot defined foo { + say "No foo yet defined"; +} + +my foo = 1; + +if defined foo { + put "foo is defined and has the value "; + say foo; +} +``` + +### Synonyms + +Each variable can have as many synonyms as wished. A synonym is another name to access the content of a specific variable. Here is an example of how to use is: + +``` +my foo = "foo"; +my bar = \foo; +foo = "bar"; + +# The synonym variable should now also set to "bar" +assert "bar" == bar; +``` + +Synonyms can be used for all kind of identifiers. It's not limited to normal variables but can be also used for function and procedure names etc (more about functions and procedures later). + +``` +# Create a new procedure baz +proc baz { say "I am baz"; } + +# Make a synonym baz, and undefine baz +my bay = \baz; + +undef baz; + +# bay still has a reference of the original procedure baz +bay; # this prints aut "I am baz" +``` + +The "syms" keyword gives you the total number of synonyms pointing to a specific value: + +``` +my foo = 1; +say syms foo; # Prints 1 + +my baz = \foo; +say syms foo; # Prints 2 +say syms baz; # Prints 2 + +undef baz; +say syms foo; # Prints 1 +``` + +## Statements and expressions + +A Fype program is a list of statements. Each keyword, expression or function call is part of a statement. Each statement is ended with a semicolon. Example: + +``` +my bar = 3, foo = 1 + 2; +say foo; +exit foo - bar; +``` + +### Parenthesis + +All parenthesis for function arguments are optional. They help to make the code better readable. They also help to force precedence of expressions. + +### Basic expressions + +Any "any" value holding a string will be automatically converted to an integer value. + +``` +(any) <any> + <any> +(any) <any> - <any> +(any) <any> * <any> +(any) <any> / <any> +(integer) <any> == <any> +(integer) <any> != <any> +(integer) <any> <= <any> +(integer) <any> gt <any> +(integer) <any> <> <any> +(integer) <any> gt <any> +(integer) not <any> +``` + +### Bitwise expressions + +``` +(integer) <any> :< <any> +(integer) <any> :> <any> +(integer) <any> and <any> +(integer) <any> or <any> +(integer) <any> xor <any> +``` + +### Numeric expressions + +``` +(number) neg <number> +``` + +... returns the negative value of "number": + +``` +(integer) no <integer> +``` + +... returns 1 if the argument is 0, otherwise it will return 0! If no argument is given, then 0 is returned! + +``` +(integer) yes <integer> +``` + +... always returns 1. The parameter is optional. Example: + +``` +# Prints out 1, because foo is not defined +if yes { say no defined foo; } +``` + +## Control statements + +Control statements available in Fype: + +``` +if <expression> { <statements> } +``` + +... runs the statements if the expression evaluates to a true value. + +``` +ifnot <expression> { <statements> } +``` + +... runs the statements if the expression evaluates to a false value. + +``` +while <expression> { <statements> } +``` + +... runs the statements as long as the expression evaluates to a true value. + +``` +until <expression> { <statements> } +``` + +... runs the statements as long as the expression evaluates to a false value. + +## Scopes + +A new scope starts with an { and ends with an }. An exception is a procedure, which does not use its own scope (see later in this manual). Control statements and functions support scopes. The "scope" function prints out all available symbols at the current scope. Here is a small example: + +``` +my foo = 1; + +{ + # Prints out 1 + put defined foo; + { + my bar = 2; + + # Prints out 1 + put defined bar; + + # Prints out all available symbols at this + # point to stdout. Those are: bar and foo + scope; + } + + # Prints out 0 + put defined bar; + + my baz = 3; +} + +# Prints out 0 +say defined bar; +``` + +Another example including an actual output: + +``` +./fype -e ’my global; func foo { my var4; func bar { my var2, var3; func baz { my var1; scope; } baz; } bar; } foo;’ +Scopes: +Scope stack size: 3 +Global symbols: +SYM_VARIABLE: global (id=00034, line=-0001, pos=-001, type=TT_INTEGER, dval=0.000000, refs=-1) +SYM_FUNCTION: foo +Local symbols: +SYM_VARIABLE: var1 (id=00038, line=-0001, pos=-001, type=TT_INTEGER, dval=0.000000, refs=-1) +1 level(s) up: +SYM_VARIABLE: var2 (id=00036, line=-0001, pos=-001, type=TT_INTEGER, dval=0.000000, refs=-1) +SYM_VARIABLE: var3 (id=00037, line=-0001, pos=-001, type=TT_INTEGER, dval=0.000000, refs=-1) +SYM_FUNCTION: baz +2 level(s) up: +SYM_VARIABLE: var4 (id=00035, line=-0001, pos=-001, type=TT_INTEGER, dval=0.000000, refs=-1) +SYM_FUNCTION: bar +``` + +## Definedness + +``` +(integer) defined <identifier> +``` + +... returns 1 if "identifier" has been defined. Returns 0 otherwise. + +``` +(integer) undef <identifier> +``` + +... tries to undefine/delete the "identifier". Returns 1 if it succeeded, otherwise 0 is returned. + +## System + +These are some system and interpreter specific built-in functions supported: + +``` +(void) end +``` + +... exits the program with the exit status of 0. + +``` +(void) exit <integer> +``` + +... exits the program with the specified exit status. + +``` +(integer) fork +``` + +... forks a subprocess. It returns 0 for the child process and the pid of the child process otherwise! Example: + +``` +my pid = fork; + +if pid { + put "I am the parent process; child has the pid "; + say pid; + +} ifnot pid { + say "I am the child process"; +} +``` + +To execute the garbage collector do: + +``` +(integer) gc +``` + +It returns the number of items freed! You may wonder why most of the time it will return a value of 0! Fype tries to free not needed memory ASAP. This may change in future versions in order to gain faster execution speed! + +### I/O + +``` +(any) put <any> +``` + +... prints out the argument + +``` +(any) say <any> +``` + +is the same as put, but also includes an ending newline. + +``` +(void) ln +``` + +... just prints a newline. + +## Procedures and functions + +### Procedures + +A procedure can be defined with the "proc" keyword and deleted with the "undef" keyword. A procedure does not return any value and does not support parameter passing. It's using already defined variables (e.g. global variables). A procedure does not have its own namespace. It's using the calling namespace. It is possible to define new variables inside of a procedure in the current namespace. + +``` +proc foo { + say 1 + a * 3 + b; + my c = 6; +} + +my a = 2, b = 4; + +foo; # Run the procedure. Print out "11\n" +say c; # Print out "6\n"; +``` + +### Nested procedures + +It's possible to define procedures inside of procedures. Since procedures don't have its own scope, nested procedures will be available to the current scope as soon as the main procedure has run the first time. You may use the "defined" keyword in order to check if a procedure has been defined or not. + +``` +proc foo { + say "I am foo"; + + undef bar; + proc bar { + say "I am bar"; + } +} + +# Here bar would produce an error because +# the proc is not yet defined! +# bar; + +foo; # Here the procedure foo will define the procedure bar! +bar; # Now the procedure bar is defined! +foo; # Here the procedure foo will redefine bar again! +``` + +### Functions + +A function can be defined with the "func" keyword and deleted with the "undef" keyword. Function do not yet return values and do not yet supports parameter passing. It's using local (lexical scoped) variables. If a certain variable does not exist, when It's using already defined variables (e.g. one scope above). + +``` +func foo { + say 1 + a * 3 + b; + my c = 6; +} + +my a = 2, b = 4; + +foo; # Run the procedure. Print out "11\n" +say c; # Will produce an error, because c is out of scoped! +``` + +### Nested functions + +Nested functions work the same way the nested procedures work, with the exception that nested functions will not be available anymore after the function has been left! + +``` +func foo { + func bar { + say "Hello i am nested"; + } + + bar; # Calling nested +} + +foo; +bar; # Will produce an error, because bar is out of scope! +``` + +## Arrays + +Some progress on arrays has been made too. The following example creates a multi dimensional array "foo". Its first element is the return value of the func which is "bar". The fourth value is a string ”3” converted to a double number. The last element is an anonymous array which itself contains another anonymous array as its last element: + +``` +func bar { say ”bar” } +my foo = [bar, 1, 4/2, double ”3”, [”A”, [”BA”, ”BB”]]]; +say foo; +``` + +It produces the following output: + +``` +% ./fype arrays.fy +bar +01 +2 +3.000000 +A +BA +BB +``` + +## Fancy stuff + +Fancy stuff like OOP or Unicode or threading is not planed. But fancy stuff like function pointers and closures may be considered.:) + +## May the source be with you + +You can find all of this on the GitHub page. There is also an "examples" folders containing some Fype scripts! + +=> https://github.com/snonux/fype + +E-Mail me your thoughts at comments@mx.buetow.org! diff --git a/content/gemtext/gemfeed/atom.xml b/content/gemtext/gemfeed/atom.xml index b5fbe934..2a97e1f2 100644 --- a/content/gemtext/gemfeed/atom.xml +++ b/content/gemtext/gemfeed/atom.xml @@ -1,6 +1,6 @@ <?xml version="1.0" encoding="utf-8"?> <feed xmlns="http://www.w3.org/2005/Atom"> - <updated>2021-05-03T10:06:42+01:00</updated> + <updated>2021-05-07T09:45:39+01:00</updated> <title>buetow.org feed</title> <subtitle>Having fun with computers!</subtitle> <link href="gemini://buetow.org/gemfeed/atom.xml" rel="self" /> @@ -11,65 +11,1092 @@ <link href="gemini://buetow.org/gemfeed/2021-04-24-welcome-to-the-geminispace.gmi" /> <id>gemini://buetow.org/gemfeed/2021-04-24-welcome-to-the-geminispace.gmi</id> <updated>2021-04-24T19:28:41+01:00</updated> - <summary>Have you reached this article already via Gemini? You need a special client for that, web browsers such as Firefox, Chrome, Safari etc. don't support the Gemini protocol. The Gemini address of this site (or the address of this capsule as people say in Geminispace) is: ... to read on visit my site.</summary> <author> <name>Paul Buetow</name> <email>comments@mx.buetow.org</email> </author> + <summary>Have you reached this article already via Gemini? You need a special client for that, web browsers such as Firefox, Chrome, Safari etc. don't support the Gemini protocol. The Gemini address of this site (or the address of this capsule as people say in Geminispace) is: ... to read on visit my site.</summary> + <content type="xhtml"> + <div xmlns="http://www.w3.org/1999/xhtml"> + <h1>Welcome to the Geminispace</h1> +<p>Have you reached this article already via Gemini? You need a special client for that, web browsers such as Firefox, Chrome, Safari etc. don't support the Gemini protocol. The Gemini address of this site (or the address of this capsule as people say in Geminispace) is:</p> +<a class="textlink" href="gemini://buetow.org">gemini://buetow.org</a><br /> +<p>If you however still use HTTP then you are just surfing the fallback HTML version of this capsule. In that case I suggest reading on what this is all about :-).</p> +<pre> + + /\ + / \ + | | + |NASA| + | | + | | + | | + ' ` + |Gemini| + | | + |______| + '-`'-` . + / . \'\ . .' + ''( .'\.' ' .;' +'.;.;' ;'.;' ..;;' AsH + +</pre> +<h2>Motivation</h2> +<h3>My urge to revamp my personal website</h3> +<p>For some time I had to urge to revamp my personal website. Not to update the technology and the design of it but to update all the content (+ keep it current) and also to start a small tech blog again. So unconsciously I started to search for a good platform and/or software to do all of that in a KISS (keep it simple & stupid) way.</p> +<h3>My still great Laptop running hot</h3> +<p>Earlier this year (2021) I noticed that my 6 year old but still great Laptop started to become hot and slowed down while surfing the web. Also, the Laptop's fan became quite noisy. This is all due to the additional bloat such as JavaScript, excessive use of CSS, tracking cookies+pixels, ads and so on there was on the website. </p> +<p>All what I wanted was to read an interesting article but after a big advertising pop-up banner appeared and made everything worse I gave up and closed the browser tab.</p> +<h2>Discovering the Gemini internet protocol</h2> +<p>Around the same time I discovered a relatively new more lightweight protocol named Gemini which does not support all these CPU intensive features like HTML, JavaScript and CSS do. Also, tracking and ads is not supported by the Gemini protocol.</p> +<p>The "downside" is that due to the limited capabilities of the Gemini protocol all sites look very old and spartan. But that is not really a downside, that is in fact a design choice people made. It is up to the client software how your capsule looks. For example, you could use a graphical client with nice font renderings and colors to improve the appearance. Or you could just use a very minimalistic command line black-and-white Gemini client. It's your (the user's) choice.</p> +<i>Screenshot Amfora Gemini terminal client surfing this site:</i><a href="https://buetow.org/gemfeed/2021-04-24-welcome-to-the-geminispace/amfora-screenshot.png"><img alt="Screenshot Amfora Gemini terminal client surfing this site" title="Screenshot Amfora Gemini terminal client surfing this site" src="https://buetow.org/gemfeed/2021-04-24-welcome-to-the-geminispace/amfora-screenshot.png" /></a><br /> +<p>Why is there a need for a new protocol? As the modern web is a superset of Gemini, can't we just use simple HTML 1.0? That's a good and valid question. It is not a technical problem but a human problem. We tend to abuse the features once they are available. You can be sure that things stay simple and efficient as long as you are using the Gemini protocol. On the other hand you can't force every website in the modern web to only create plain and simple looking HTML pages.</p> +<h2>My own Gemini capsule</h2> +<p>As it is very easy to set up and maintain your own Gemini capsule (Gemini server + content composed via the Gemtext markup language) I decided to create my own. What I really like about Gemini is that I can just use my favorite text editor and get typing. I don't need to worry about the style and design of the presence and I also don't have to test anything in ten different web browsers. I can only focus on the content! As a matter of fact, I am using the Vim editor + it's spellchecker + auto word completion functionality to write this. </p> +<h2>Advantages summarised</h2> +<ul> +<li>Supports an alternative to the modern bloated web</li> +<li>Easy to operate and easy to write content</li> +<li>No need to worry about various web browser compatibilities</li> +<li>It's the client's responsibility how the content is designed+presented</li> +<li>Lightweight (although not as lightweight as the Gopher protocol)</li> +<li>Supports privacy (no cookies, no request header fingerprinting, TLS encryption)</li> +<li>Fun to play with (it's a bit geeky yes, but a lot of fun!)</li> +</ul> +<h2>Dive into deep Gemini space</h2> +<p>Check out one of the following links for more information about Gemini. For example, you will find a FAQ which explains why the protocol is named "Gemini". Many Gemini capsules are dual hosted via Gemini and HTTP(S), so that people new to Gemini can sneak peek the content with a normal web browser. As a matter of fact, some people go as far as tri-hosting all their content via HTTP(S), Gemini and Gopher.</p> +<a class="textlink" href="gemini://gemini.circumlunar.space">gemini://gemini.circumlunar.space</a><br /> +<a class="textlink" href="https://gemini.circumlunar.space">https://gemini.circumlunar.space</a><br /> +<p>E-Mail me your thoughts at comments@mx.buetow.org!</p> + </div> + </content> </entry> <entry> <title>DTail - The distributed log tail program</title> <link href="gemini://buetow.org/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.gmi" /> <id>gemini://buetow.org/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.gmi</id> <updated>2021-04-22T19:28:41+01:00</updated> - <summary>This article first appeared at the Mimecast Engineering Blog but I made it available here in my personal Gemini capsule too. ...to read on visit my site.</summary> <author> <name>Paul Buetow</name> <email>comments@mx.buetow.org</email> </author> + <summary>This article first appeared at the Mimecast Engineering Blog but I made it available here in my personal Gemini capsule too. ...to read on visit my site.</summary> + <content type="xhtml"> + <div xmlns="http://www.w3.org/1999/xhtml"> + <h1>DTail - The distributed log tail program</h1> +<i>DTail logo image:</i><a href="https://buetow.org/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program/dtail-logo.png"><img alt="DTail logo image" title="DTail logo image" src="https://buetow.org/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program/dtail-logo.png" /></a><br /> +<p>This article first appeared at the Mimecast Engineering Blog but I made it available here in my personal Gemini capsule too.</p> +<a class="textlink" href="https://medium.com/mimecast-engineering/dtail-the-distributed-log-tail-program-79b8087904bb">Original Mimecast Engineering Blog post at Medium</a><br /> +<p>Running a large cloud-based service requires monitoring the state of huge numbers of machines, a task for which many standard UNIX tools were not really designed. In this post, I will describe a simple program, DTail, that Mimecast has built and released as Open-Source, which enables us to monitor log files of many servers at once without the costly overhead of a full-blown log management system.</p> +<p>At Mimecast, we run over 10 thousand server boxes. Most of them host multiple microservices and each of them produces log files. Even with the use of time series databases and monitoring systems, raw application logs are still an important source of information when it comes to analysing, debugging, and troubleshooting services.</p> +<p>Every engineer familiar with UNIX or a UNIX-like platform (e.g., Linux) is well aware of tail, a command-line program for displaying a text file content on the terminal which is also especially useful for following application or system log files with tail -f logfile.</p> +<p>Think of DTail as a distributed version of the tail program which is very useful when you have a distributed application running on many servers. DTail is an Open-Source, cross-platform, fairly easy to use, support and maintain log file analysis & statistics gathering tool designed for Engineers and Systems Administrators. It is programmed in Google Go.</p> +<h2>A Mimecast Pet Project</h2> +<p>DTail got its inspiration from public domain tools available already in this area but it is a blue sky from-scratch development which was first presented at Mimecast’s annual internal Pet Project competition (awarded with a Bronze prize). It has gained popularity since and is one of the most widely deployed DevOps tools at Mimecast (reaching nearly 10k server installations) and many engineers use it on a regular basis. The Open-Source version of DTail is available at:</p> +<a class="textlink" href="https://dtail.dev">https://dtail.dev</a><br /> +<p>Try it out — We would love any feedback. But first, read on…</p> +<h2>Differentiating from log management systems</h2> +<p>Why not just use a full-blown log management system? There are various Open-Source and commercial log management solutions available on the market you could choose from (e.g. the ELK stack). Most of them store the logs in a centralized location and are fairly complex to set up and operate. Possibly they are also pretty expensive to operate if you have to buy dedicated hardware (or pay fees to your cloud provider) and have to hire support staff for it.</p> +<p>DTail does not aim to replace any of the log management tools already available but is rather an additional tool crafted especially for ad-hoc debugging and troubleshooting purposes. DTail is cheap to operate as it does not require any dedicated hardware for log storage as it operates directly on the source of the logs. It means that there is a DTail server installed on all server boxes producing logs. This decentralized comes with the direct advantages that there is no introduced delay because the logs are not shipped to a central log storage device. The reduced complexity also makes it more robust against outages. You won’t be able to troubleshoot your distributed application very well if the log management infrastructure isn’t working either.</p> +<i>DTail sample session animated gif:</i><a href="https://buetow.org/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program/dtail.gif"><img alt="DTail sample session animated gif" title="DTail sample session animated gif" src="https://buetow.org/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program/dtail.gif" /></a><br /> +<p>As a downside, you won’t be able to access any logs with DTail when the server is down. Furthermore, a server can store logs only up to a certain capacity as disks will fill up. For the purpose of ad-hoc debugging, these are not typically issues. Usually, it’s the application you want to debug and not the server. And disk space is rarely an issue for bare metal and VM-based systems these days, with sufficient space for several |