Introduction

The GNU Hurd is a modular based operating kernel built on top the micro-kernel GNU Mach. The Hurd has numerous advantages over the traditional and monolithic design of a Unix kernel. Various parts of the Hurd can be separated, loaded, unloaded, or replaced, in ways that monolithic kernels, like Linux cannot replicate. If one part of the Hurd crashes, it won't crash the rest of the kernel. It's also much smaller, elegant, and forward looking than Linux.

The Linux kernel is close to 15 or 16 million lines of code! It is a massive black hole, that is only getting bigger and harder to maintain. In contrast, the Hurd is smaller and is designed so that it can be written with in languages other than C. Applications can mount and start their own filesystem on the fly, and it is designed to lower the barrier of entry for young hackers kernel hackers.

Since Linux is monolithic, a small change can affect a very different part of the kernel in significant ways. This is why developers occasionally are discouraged from changing the existing code base. On the Hurd, each server runs as a separate userspace process, which means new servers cannot easily affect, damage, or introduce bugs to the existing Hurd. The Hurd even goes a step further and lets the user modify any existing server or use their own instead.

The Hurd is also more secure than a traditional Unix kernel. A traditional unix user has certain permissions to read, write, or execute files they control. Any time a user runs a program, traditionally a unix kernel runs that program with the full permissions of the original user. This means any program that runs, can do things that the user could do! Why should Firefox have the ability to access your gpg keys? Why should gthumb be able to send email on your behalf? Linux is actively trying to fix this problem, but many vulnerabilities still exist. In contrast, the Hurd is designed to be a capability based system. Applications run with the fewest permissions possible.

While there are numerous technological advantages to the Hurd, many of them have not been implemented yet. The Hurd suffers from lack of funding and developers to extend it, but it is still an attractive platform to learn kernel hacking. The Hurd allows one to modify running kernel servers, without affecting other users of the system without rebooting. If a kernel server is running slowly or poorly, one can attaching GDB to the running process and debug it! This is something that Linux cannot easily be modified to do.

Documentation things

dde is not really being maintained. These are drivers from the linux kernel. The Rump kernel seems to be a better option.

Important programs to use

settrans

settrans [option] node [translator arg] Settrans allows one to set a translator on a file or a directory. If no translator is specified, then the existing translator on the node is cleared.

-a	Start translator and set it as node's active translator
-c	create the node if it doesn't exist
-C	chroot stuff.
-F	starting a translator write its pid to a file
-L	if a file already has a translator, put the new one on top of it
-o	disconnect the old translator from the filesystem
-p	make the translator persistant across reboots
-P	pause, don't start the translator until newline
	this gives you time to start the debugger on a translator
-s	start the translator as active
--stack	replace a translator
-t	timeout for translator startup time
-U
-x	only set the translator if there is not there one already

	If you are setting a passive translator, and node already has
	an active translator, then these options apply
-g	go away
-k	keep the translator running

	if there is an active translator
-f	force the translator to shutdown
-R	shutdown the translators beneath this one too
-S	don't sync before killing
-?	give some help

By default a passive translator is set

showtrans

fsysopts

Shows the passive translators on a node. Set the command line arguments for translators. This lets you modify currently running translators on the fly!

I can use this to create a completed file for nginx. I'll use either stowfs or unionfs to create a complete file. Then when I add a new user, I'll use fsysopts to change the translator arguments.

fsysopts [option...] filesys [fs_option...]

This is actually quite cool! Let's set up a file called hello. Mount the hello translator on it, and then modify what the hello translator says!

$ touch hello $ settrans hello /hurd/hello $ showtrans hello /hurd/hello $ cat ./hello "Hello, World!" $ fsysopts ./hello /hurd/hello --contents='Hello, World! '

So now we know that we can modify the --contents value of the /hurd/hello translator.

$ fsysopts ./hello --contents="Hello, Joshua! " $cat ./hello Hello Joshua!

Vocab

Port things

Port

Port Bucket

Port Class

atomic operations vs. global locks

A port is a inter-process communication (IPC) channels implemented in the kernel. They are similar to UNIX pipes. Only one task can receive messages from a port. Many tasks can send messages to the same port. A port bucket is a collection of ports with some extra data and an ability to lock the bucket so the ports cannot be used to communicate with any process. Is also a collection of ports, each of which can be conveniently manipulated. A port class is not required to share any ports with a bucket and vice versa.

Global locks are one way to program via threads. Some lock ensures that one thread has write access to some variable. The other thread must wait until the lock is released.

Atomic operations is a way to write threads without using locks. It is a way to guarantee that no write operation will cause a data race. Essentially atomic operations ensure that a write operation occurs in one step relative to other threads. It means that no thread can view the value while it is being changed. I think it means that the write must be one line of assembly.

File

It sounds like a way or programming Every file in the Hurd can be accessed via a port. When a user wishes to access a file, it makes a remote procedure call to the server implementing the filesystem.

Capabilities things

Capabilities

so file_t is actually mach_port_t. A capability is a process' right to perform various actions to another object. A capability can be thought of as a car key. The car key doesn't care who has the key. If you have the key, then you can drive the car, listen to music, etc.

The Hurd has capabilities implemented as Mach ports. BUT these ports are not persistent. So when you reboot all those capabilities are lost. To work around it, ACLs (Access Control Lists) are used to figure out who has permissions to do what. For example my current home directory looks like


ls -l

total 108 drwxr-xr-x 4 joshua home 4096 May 12 20:52 config-files drwxr-xr-x 2 joshua home 4096 May 12 20:44 Desktop drwxr-xr-x 3 joshua home 4096 Jun 8 14:55 documents drwx------ 2 joshua home 4096 Jun 5 22:18 Downloads drwxr-xr-x 19 joshua home 4096 Jun 6 10:19 music drwxr-xr-x 2 joshua home 4096 Jun 7 13:33 videos

Each node in the filesystem begins with lots of "r"s, "w"s, and "x"s. These specify which users can read, write, or execute those nodes in the filesystem.

Capability operating system

Every process has some capabilities. They have the least amount of capabilities in order to run. These capabilities can be passed around so that other programs can have the same capabilities as other programs.

Modern OSs like UNIX, have the https://en.wikipedia.org/wiki/Confused_deputy_problem where one program tricks another. This can only happen in an operating system that has Access control lists, like UNIX, but in a capability based system, this cannot happen.

Unfortunately the Hurd is not a true capability operating system. To see how it might become a capability system, read the link.

http://lists.gnu.org/archive/html/l4-hurd/2005-10/msg00081.html

Principle of least privilege

https://en.wikipedia.org/wiki/Object-capability_model Principle of least privilege, means that every program runs with the least amount of privileges it needs in order to run.

ABAC

This is a security principle. Firefox does not need to be able to access your gpg key, nor does your vlc need access to your webcam? Authorization-based access control

File Descriptor

Access is granted by a server. Some program wants access to some resource and an authorizing server create a capability that allows this to happen. In the Hurd this is done with the auth server. When a process on UNIX attempts to open a file, the kernel accesses the file and returns a port. The process does not directly have access to the hard drive. So if you have 10 files open, then proc will have 10 entries like so: \\

/proc/PID/fd/ \\

Translator

Allegedly, these are like capabilities, but they do not survive reboot. A translator is a Hurd process that implements a filesystem interface. When one accesses any file on disk, a translator provides the file. Users can write their own translators that act on portions of the filesystem. For example, one could write a translator that turns an xml act like a directory. For example, suppose you have a xml file (file.xml) like so

<p>
<div id="This is a directory">
</p>
<p>
  <p id="file1"> this is a file </p>
</p>
<p>
  <p id="file2"> this is another file </p>
</p>
<p>
</div>
</p>

If you make a translator responsible for serving it, then cd file.xml; ls would produce:


file1
file2

Active Translator

Passive Translator

Short Circuiting

An active translator is one is currently operating on a filesystem node, but will be lost on reboot. A passive translator is one that is persistent after reboots. If a passive translator is set for /home/joshua/documents that connects documents to an network file system, then whenever joshua tries to access his documents, if the translator is not already running, it will start to connect to the network file system to access joshua's documents. https://www.gnu.org/software/hurd/hurd/translator/short-circuiting.html The hurd has a symlink translator. One can use it to create symlinks. However, the Hurd will be faster if the filesystem handles this. Instead of having extfs server start up a symlink server everytime it encounters a symlink, the ext2fs server can implement the task of the symlink server itself. This means we do not have to start a symlink server and save some resources as well as minimize RPCs.

Translator problems

TODO Translators set up by untrusted users

State "TODO" from [2016-06-08 Wed 16:05]

I can write up the page to change the text on the wiki to be more readable.

Suppose you have a translator that implements a softlink to another node in the filesystem without revealing that that is what it is doing. So home/User2/documents/secretDir points to /home/User1. If root does a rm -r /home/User2, then /home/User1 gets all of his data deleted!

Trust the behavior of Translators

Translator pages I should read:

https://www.gnu.org/software/hurd/hurd/translator/writing/example.html

Setting up translators - HowTo

https://www.gnu.org/software/hurd/open_issues/translators_set_up_by_untrusted_users.html https://www.gnu.org/software/hurd/open_issues/trust_the_behavior_of_translators.html https://www.gnu.org/software/hurd/hurd/translator/examples.html Additional translators can be got from incubator, or hurd-extras.

cvs -z3 -d:pserver:anonymous@cvs.savannah.nongnu.org:/sources/hurdextras co

httpfs translator

$ settrans -a tmp/ hurd/httpfs www.hurd-project.com

$ settrans -a tmp/ hurd/httpfs www.hurd-project.com --proxy= --port= $ cd tmp/ $ ls -l

?ftpfs translator

$ settrans -cgap ftp /hurd/hostmux /hurd/ftpfs / $ cd ftp ftp$ ls ftp$ cd ftp.fr.debian.org ftp/ftp.fr.debian.org $ ls

tarfs translator

You can use tarfs to mount (almost) any tar file (currently broken, 2010-08-25):

$ settrans -ca a /hurd/tarfs -z myfile.tar.gz $ settrans -ca b /hurd/tarfs -y myfile.tar.bz2 $ settrans -ca c /hurd/tarfs myfile.tar

You can even use it to create new tar files:

$ settrans -ca new /hurd/tarfs -cz newfile.tar.gz $ cp -r all my files new/ $ syncfs new

This is not as fast as tar czvf newfile.tar.gz all my files, but at least it's more original. ;)

cvsfs translator

$ settrans -ac cvsfs_testing /hurd/cvsfs cvs.savannah.nongnu.org /sources/hurdextras $ cd cvsfs_testing

pfinet translator -- configuring your network interface

$ settrans -fgca /servers/socket/2 /hurd/pfinet -i -a -m -g

Console translator -- setting up virtual consoles

$ console -d vga -d pc_mouse -d pc_kbd -d generic_speaker /dev/vcs

?iso9660fs translator -- 'mounting' your cdrom

$ settrans -ac /cdrom /hurd/iso9660fs /dev/

ext2fs translator -- 'mounting' an ext2fs partition

$ settrans -ac /linux /hurd/ext2fs /dev/

unionfs translator

To join "foo/" "bar/" and "baz/" in the directory "quux/", just do:

$ settrans -capfg quux/ hurd/unionfs foo bar/ baz/

TODO I can give more cool examples with translators

State "TODO" from [2016-06-08 Wed 16:10]

https://www.gnu.org/software/hurd/hurd/translator.html

https://www.gnu.org/software/hurd/hurd/documentation/translators.html

https://www.gnu.org/software/hurd/hurd/debugging/translator.html

protid

Virtual File System

https://www.gnu.org/software/hurd/hurd/documentation/translator_primer.html A protid is short for a protected id. So no single process serves the filesystem. Suppose a user wants to access the file home/USERNAME/Documents/resume.pdf. First the process looking for the file, sees if there is a translator mounted on . If there is not, then it tries to access /home. If there is no translator there, it tries to access /home/USERNAME. If there is no translator there, then it tries to access /home/USERNAME/Documents, etc. The reason the Hurd works like this, is because a user can easily assign a translator at any given file or directory in the filesystem.

Computer Bought the Farm

For example, consider that the user mounted an NSF filesystem (Network Filesystem) at /home/USERNAME/Documents. When a process tries to access resume.pdf, then, if the nfs translator is started and mounts the remote filesystem. When that happens the user can transparently open the remote file resume.pdf. Computer bought the farm means that a server crashed. It is not a huge issue. Just kill the translator and try again.


settrans -fg FILE

Identity Based Access Control

Access Control Lists

real time things

real time operating system

The Identity Based Access control is when permissions are given to a process based on its identity. This can be problematic if one program attempts to do something on behalf on another. If Firefox wants to access my filesystem, but uses gpg to decrypt a file, gpg now is accessing a file. BUT gpg doesn't have Firefox's ID. So how does one get gpg to access the file but ONLY have the permissions that Firefox has? This is called the Confused Deputy Problem. An access control list says that a program can access a program if it has the proper permissions based on its ID.

This means that if an operating system to scheduled to do something at a specified time, then it must guarantee that the thing will be done very nearly at the time it said it would do it. This means that the lag time must be very short. Many modern operating systems need to be real time for gaming and video players. Other wise the game/video can be choppy.

react as quickly as possible to events

realtime

keep the highest priority activity running as much as possible

An OS does things when it is scheduled to do them.

locks

mcs locks

ticket locks

stack

heap

Hurd

Things the Hurd can do

SATA drives

The Hurd now supports 128GB partition sizes

types of virtualization

subhurds

You need some global locks for this to work... Linux uses mcs locks have annoying limitations A SATA drive driver was added in 10th May 2013. It should work for many SATA drives. https://www.gnu.org/software/hurd/hurd/subhurd.html

Run two instances of the hurd ontop of the same mach. This is great for debugging purposes and probably other things.

I need to use qemu to provide a new partition that uses the same qemu image or a different one. If a use the same qemu image, then the subhurd won't be able to write any changes. So it's just better to have two different images. One for the main hurd and another for the subhurd.

I'll need to tell qemu to add the image.


-hdd FILE

Then I'll need to find the root filesystem of that extra debian image.


fdisk -l /dev/hd1

Then I'll just set the ext2fs server running like so:


su settrans -c /home/joshua/hd1s1 /hurd/ext2fs /dev/hd1s1

Then in the hurd I can just run boot /dev/hd0s6 or whatever the file name is.


boot hd1s1

BUT the terminal that runs that command, will take all new commands to run in the subhurd. So I'll need to ssh into the hurd with another terminal, then run boot. BUT that might not really work either. Because by default the subhurd does not have internet access...

remap one service for another

Hurd Servers

To exit or close the subhurd, I'll have to run halt or reboot. Some Hurd servers can be found in $(HURD)/trans/

To get help about how to use a server run


/hurd/hello --help

Usage: hello [OPTION...] A translator providing a warm greeting.

-c, --contents=STRING Specify the contents of the virtual file -?, --help Give this help list --usage Give a short usage message -V, --version Print program version

Mandatory or optional arguments to long options are also mandatory or optional for any corresponding short options.

Report bugs to .

Auth

This is apparently a Access Control Lists server. Suppose Firefox wishes to access a file on disk. Firefox requests the file from the filesystem. The filesystem wants to be sure that Firefox can do this, so it asks the Auth server. https://www.gnu.org/software/hurd/hurd/documentation/auth.html

3 things the auth server can do:

remap

If it receives two authentication ports, then the server will provide a 3rd ports that is the union of the first sets of uids and guids.
Any user with uid 0 can create a arbitrary authentication port.
Allows any two arbitrary programs to trust one another.

hurd-2017-03-17-13:28:17< teythoon> do you know how to use 'remap' to test your translator ? hurd-2017-03-17-13:28:42< teythoon> eth-multiplexer will help too hurd-2017-03-17:13:29:45< teythoon> see http://darnassus.sceen.net/~hurd-web/hurd/subhurd/#index4h2 for how to setup eth-multiplexer on your hurd machine hurd-2017-03-17-13:31:59< teythoon> then you can settrans -ac my-2 path/to/my/ip-stack -what -ever ; remap /servers/socket/2 $(pwd)/my-2 -- ping 8.8.8.8 hurd-2017-03-17:13:39:18< jlledom> I didn't know the eth-multiplexer, I configured two interfaces in my virtual machine and remap exactly that way

process server

This is an information server.

It keeps track of host information: hostname, hostid, and system version.
It takes care of POSIX sessions and session groups. Some programs will want this information.
The process server maintains a 1-1 mapping between mach tasks and hurd processes. Each process can register a port with the process server, which allows anyone to communicate to that process. That means each process must implement security stuff themselves.
It also does process collections? http://darnassus.sceen.net/~hurd-web/hurd-paper/

The process server is optional. Mach has some kind of restrictions that means a user must be root to see all the processes.

Users can have more than one uid

password

BUT now I'm not certain why that is a problem...I feel like most users wouldn't need to know all the running processes... found in /servers/password

This checks to see if a user's password is the correct password.

pfinet

symlink

It also checks to see if a group's password is the real password. This is the TCP/IP networking stack. It is using dde.

Set up a symlink. Apparently it's much faster if the filesystem does this, but you can let the symlink translator create a symlink for you. This can be useful, if you are developing a filesystem, but don't want to create a symlink yet.


settrans -a link-to-programming /hurd/symlink /home/joshua/programming/

the hurd servers parse arguments with argp. I can learn more about it here:

crash server

info:libc#Argp

You can use the crash server very cool!

https://savannah.gnu.org/bugs/?17319


cat crash.c

// this doesn't seem to actually crash c... /* int main (void) { return (int ) 0; */

* } */ / this will actually crash a c program int main () { return 1/0; }

Running the program, will create a file called core.

ls -l core

ls -l /servers/crash

Hurd translators

hello


touch hello
cat hello
settrans -a ./hello /hurd/hello
cat hello # Hello World!
settrans -g ./hello
cat hello

xmlfs

The xmlfs documentation can be found here: http://cvs.savannah.gnu.org/viewvc/*checkout*/hurdextras/xmlfs/README The example xml file can be found here: http://cvs.savannah.gnu.org/viewvc/*checkout*/hurdextras/xmlfs/example.xml

xmlfs allows one to create a directory tree out of an xml file. Cool eh?

xmlfs - a translator for accessing XML documents

This is only an alpha version. It works in read only. It supports text nodes and attributes. It doesn't do anything fancy like size computing, though. Here is an example of how to use it:

$ settrans -ca xml /hurd/xmlfs example.xml #the website says to use ./xmlfs $ cd xml; ls library0 library1 $ cd library0; ls -A .text1 .text2 @name book0 book1 book2 sub-library0 sub-library1 $ cat .text2

CDATA, again !

$ cat book0 Mark Twain 4242 $ cat book0/author/.text Mark Twain

As you can see, text nodes are named .textN, with N an integer starting from 0. Sorting is supposed to be stable, so you get the same N every time you access the same file. If there is only one text node at this level, N is ommitted. Attributes are prefixed with @.

An example file, example.xml, is provided. Of course, it does not contain anything useful. xmlfs has been tested on several-megabytes XML documents, though.

Comments are welcome.

stowfs

run

unionfs

Hurd Libraries

-- Manuel Menal One can create an arbitrary directory tree on the fly. This just outputs the result of running a program.

Trivfs

This is the library that is used to create simple translators that operate on single filesystem nodes.

It has some weird callback function names though:

Real basic trivfs stuff

trivfs_S_io_read

redefine extern struct arpg * trivfs_runtime_argp to add option parsing to a trivfs program.

After I run argp, I should check to see if my translator was run as a program or started via settrans.

various functions and variables that must be defined

extern int trivfs_fstype usually FSTYPE_MISC

extern int trivfs_fsid
extern int trivfs_allow_open

extern int trivfs_support_read
extern int trivfs_support_write
extern int trivfs_support_exec

Trivfs has some options. Read about it on p. 35 of the hurd reference manual.

libnetfs

void trivfs_modify_stat
error_t trivfs_goaway

libdiskfs

libports

libstore

libiohelp

libtreefs is this still defunct?

libthreads

libpthread

libhurduser

fshelp

libihash

libps

libshouldbeinlibc

libpipe

libpcap doesn't work on the hurd? What's that?

The Hurd Interfaces

Hurd functions

io_stat

file_chown

file_chauthor

file_chmod

Writing C programs in the Hurd

This is how ftpfs, nfs, and unionfs are written. a library used for making filesystems where the translator does control the underlying data. Functions for working with ports. It's an abstraction of what the hurd expects from a message passing system. A low level interface to a fixed block or storage, which can be read or written to. A library for helping servers with io. The c thread library the p thread library A library to help with the file protocol. integer-keyed hash table library to print process information A library of functions that should be in libc The Hurd provides interfaces so that one can create remote procedure calls. Most libc functions are translated into GNU/Hurd RPCs. is an RPC If you want your program to run on the Hurd, then the top of your file needs to be:


#define _GNU_SOURCE 1

You can compile a program with gcc -g -o <program name> <program file>. For example, gcc -g -o cat cat.c.

Alternatively, you can compile programs via

gcc -g -D_GNU_SOURCE -o <program name> <program file>

Read the hurd *defs. These are functions that you can call.


ls *defs

auth.defs hurd_types.defs msg_request.defs auth_reply.defs ifsock.defs newterm.defs auth_request.defs iioctl.defs password.defs crash.defs interrupt.defs pfinet.defs crash_reply.defs io.defs process.defs default_pager.defs io_reply.defs process_reply.defs default_pager_reply.defs io_request.defs process_request.defs exec.defs ioctls.defs socket.defs exec_startup.defs kdioctl.defs startup.defs fs.defs kernel_boot.defs startup_notify.defs fs_notify.defs login.defs startup_reply.defs fsys.defs msg.defs term.defs fsys_reply.defs msg_reply.defs tioctl.defs

read the navigating document


cat navigating

Some ideas on navigating and understanding the Hurd source code.

First you must understand Mach IPC and MiG. Pay special attention to the way that various kinds of MiG calls manage memory; this is crucial to avoid leaks. The "Mach server writers' guide" explains all this.

Most programs that make up the Hurd are built out of libraries; a solid understanding of the libraries that make up the source is essential. First start by reading the libports library specification (in libports/ports.h). This library manages the Mach ports that servers handle.

Then start looking at some Hurd interfaces. A good place to start is to look at the proc server. There is only one proc server in a running system, but examine the way the interface (hurd/process.defs) is written and the way the proc server implements it.

Then look at the auth server; make sure you understand how an auth transaction works. Again, by looking at the implementation, you can see a simple example.

Filesystems are more complex; the interface specification is in hurd/io.defs and hurd/fs.defs. These interfaces are implemented by three different libraries: trivfs, diskfs, and netfs. trivfs implements single-node filesystems (that thus have no directories). Most trivfs filesystems don't even do any filesystem stuff at all. See, for example, the null translator (trans/null.c) for a simple example of using trivfs.

diskfs is used for disk-based filesystems, with one in existence now: ext2fs. If you write another diskfs-based filesystem, you should DEFINITELY imitate the algorithms found in ext2fs; this is crucial to getting locking right.

netfs is used for nfs and other such things: with directories, and all the actual filesystem operations being done by some other program (not necessarily over a network). The nfs implementation is fairly easy to understand.

Examine some translators in the trans directory to see various simple examples.

Also very important is to acquire familiarity with the Hurd and Mach calls provided in the GNU C library. Look at the header files in libc to see what they are, and read through them. Also examine parts of the libc implementation whenever you have any doubt about what an interface call should do: find where the C library uses that call, and that should help. It's worth, in fact, spending some time just exploring the implementation of various things in the hurd C library.

You should take a look at all the libraries in the Hurd; spend time reading the code. Feel free to ask questions to help understand what you read.

filesystem

Get the size of a file


#define _GNU_SOURCE 1

mach_msg_type_number_t amount;
error_t err;

err = io_readable (f, &amount);

read a file


#define _GNU_SOURCE 1

mach_msg_type_number_t amount;
error_t err;

err = io_read (f, &buf, &amount, -1, amount);

how a program is run in the hurd

rpctrace

A program is run via the file_exec RPC. The fileserver verifies that the user can run the program (via auth probably), then modify some ports for setuid execution, then invoke the standard execserver found in /servers/exec. compile a translator

gcc -g -Wall -D_GNU_SOURCE -o looper looper.c touch node

Set a translator. settrans -a node bin/rpctrace ./looper

simple malicious translator that could break stuff:

Then running the translator should trigger rpctrace.

https://www.sceen.net/the-trust-tree/

A user could create a translator at a file in /tmp. A program meant to clean /tmp, would try to remove the file, but the translator would just not properly return from a trivfs function it had recorded.

youpi ported the hurd to xen for driver stuff

teythoon hurd test? link?

This could cause the /tmp cleaner to crash, which could potentially stop other programs from using /tmp. hurd-2017-02-28-15:47:30< teythoon> the reason why youpi has focussed on xen is driver support hurd-2017-02-28-15:47:31< ram_> I'm working on compiling from source right now, and getting a few simple patches in. hurd-2017-02-28-15:48:10< ram_> How does xen driver support help the grand scheme of things? hurd-2017-02-28:15:48:27< teythoon> we have a bunch of Linux2.0 drivers in gnumach that will not work on amd64 hurd-2017-02-28-15:48:55< teythoon> with the xen port we don't need them, as we use xens faacilities hurd-2017-02-28-15:50:13< ram_> I'll have to read up on that. So what about Intel x86-64 support? That's not something you guys want? hurd-2017-02-28-15:50:15< youpi> well, it's not really the reason why I started with the xen 64bit port hurd-2017-02-28-15:50:31< youpi> it's just that it skips a lot of 64bit initialization stuff hurd-2017-02-28-15:50:44< youpi> the very task of booting from grub to a 64bit kernel is not trivial :) hurd-2017-02-28-15:50:52< youpi> so it was easier to just boot from xen hurd-2017-02-28-17:50:13< teythoon> fwiw, i have https://bitbucket.org/teythoon/hurdtest that i still use

grep-ing to the irc stuff:

the hurd uses multiplexs over lots of threads...not the way mach was intended to be used:

make the Hurd fsck etx2 by default

Debugging

debugging Translators

hurd-2016-12-10-13:59:28< braunr> not enough to really know about the project though hurd-2016-12-10-13:59:34< braunr> so i couldn't answer teythoon hurd-2016-12-10-14:21:17< teythoon> i was a bit jealous... must have been nice :) hurd-2016-12-10-14:22:24< braunr> not really hurd-2016-12-10-14:22:48< braunr> it would have been if i could get what was going on :) hurd-2016-12-10-14:24:19< teythoon> i mean meeting to work on something together is nice hurd-2016-12-10-14:24:33< teythoon> which reminds me, who is going to fosdem ? hurd-2016-12-10-14:25:00< braunr> not this year hurd-2016-12-10-14:25:02< braunr> probably the next hurd-2016-12-15-17:37:04< braunr> i'm pretty sure that i read something saying the original devs wanted sync RPCs, and didn't care much about whether IPCs were sync or not hurd-2016-12-15-17:37:34< braunr> hat's also the recommendation from mach docs iirc hurd-2016-12-15-17:37:34< braunr> that* hurd-2016-12-15-17:37:45< braunr> which is why most RPCs are declared as routines hurd-2016-12-15-17:39:12< antrik> it combines the complexity of async IPC with the complexity of threading hurd-2016-12-15-17:41:26< braunr> i don't think it does hurd-2016-12-15-17:41:45< braunr> from the point of view of the hurd developer writing his server, IPC can be considered sycnrhonous hurd-2016-12-15-17:42:22< braunr> debugging is sometimes affected hurd-2016-12-15-17:43:54< antrik> well, the problem is exacerbated by the fact that the Hurd doesn't use a thread per server-side object (which seems to be the idiom intended by the Mach developers), but rather multiplexes incoming requests over a pool of threads not bound to particular objects hurd-2016-12-15-17:46:07< braunr> that, i agree hurd-2016-12-15:17:46:28< braunr> in x15, i intend to make server thread bind to client threads hurd-2016-12-15-17:46:37< braunr> through something i would call "thread binding" hurd-2016-12-15-17:46:51< antrik> if you have a thread per object, it probably makes sense to pretend RPCs are sync (though in this case it could be argued that they should rather be in separate processes, like in EROS...) hurd-2016-12-15-17:47:12< braunr> yes, a process per client/object also fits nicely hurd-2016-12-15-17:47:20< braunr> especially when we can move receive rights hurd-2016-12-15-17:47:36< antrik> however, in a multiplexed model like in the Hurd, using threading instead of an event-drived model makes no sense hurd-2016-12-15-17:48:22< antrik> (of course, the fact that FS servers rely heavily on transparent I/O through memory mapping makes this tricky...) hurd-2016-12-15-17:48:32< braunr> no i still disagree with that hurd-2016-12-15-17:48:37< braunr> threading makes sense hurd-2016-12-15-17:48:44< braunr> mapping threads to messages doesn't hurd-2016-12-15-17:49:18< antrik> I'm talking about having a thread per request hurd-2016-12-10-13:59:28< braunr> not enough to really know about the project though hurd-2016-12-10-13:59:34< braunr> so i couldn't answer teythoon hurd-2016-12-10-14:21:17< teythoon> i was a bit jealous... must have been nice :) hurd-2016-12-10-14:22:24< braunr> not really hurd-2016-12-10-14:22:48< braunr> it would have been if i could get what was going on :) hurd-2016-12-10-14:24:19< teythoon> i mean meeting to work on something together is nice hurd-2016-12-10-14:24:33< teythoon> which reminds me, who is going to fosdem ? hurd-2016-12-10-14:25:00< braunr> not this year hurd-2016-12-10-14:25:02< braunr> probably the next hurd-2016-12-15-17:37:04< braunr> i'm pretty sure that i read something saying the original devs wanted sync RPCs, and didn't care much about whether IPCs were sync or not hurd-2016-12-15-17:37:34< braunr> hat's also the recommendation from mach docs iirc hurd-2016-12-15-17:37:34< braunr> that* hurd-2016-12-15-17:37:45< braunr> which is why most RPCs are declared as routines hurd-2016-12-15-17:39:12< antrik> it combines the complexity of async IPC with the complexity of threading hurd-2016-12-15-17:41:26< braunr> i don't think it does hurd-2016-12-15-17:41:45< braunr> from the point of view of the hurd developer writing his server, IPC can be considered sycnrhonous hurd-2016-12-15-17:42:22< braunr> debugging is sometimes affected hurd-2016-12-15-17:43:54< antrik> well, the problem is exacerbated by the fact that the Hurd doesn't use a thread per server-side object (which seems to be the idiom intended by the Mach developers), but rather multiplexes incoming requests over a pool of threads not bound to particular objects hurd-2016-12-15-17:46:07< braunr> that, i agree hurd-2016-12-15:17:46:28< braunr> in x15, i intend to make server thread bind to client threads hurd-2016-12-15-17:46:37< braunr> through something i would call "thread binding" hurd-2016-12-15-17:46:51< antrik> if you have a thread per object, it probably makes sense to pretend RPCs are sync (though in this case it could be argued that they should rather be in separate processes, like in EROS...) hurd-2016-12-15-17:47:12< braunr> yes, a process per client/object also fits nicely hurd-2016-12-15-17:47:20< braunr> especially when we can move receive rights hurd-2016-12-15-17:47:36< antrik> however, in a multiplexed model like in the Hurd, using threading instead of an event-drived model makes no sense hurd-2016-12-15-17:48:22< antrik> (of course, the fact that FS servers rely heavily on transparent I/O through memory mapping makes this tricky...) hurd-2016-12-15-17:48:32< braunr> no i still disagree with that hurd-2016-12-15-17:48:37< braunr> threading makes sense hurd-2016-12-15-17:48:44< braunr> mapping threads to messages doesn't hurd-2016-12-15-17:49:18< antrik> I'm talking about having a thread per request hurd-2017-01-10-13:13:38< braunr> we tend to set FSCKFIX=yes in /etC/default/rcS

In order to debug translators and being able to step into glibc during it, on Debian you need the hurd-dbg and libc0.3-dbg packages installed. If you need to debug the initialization of the translator, start the translator like

$ settrans -Pa /foo /usr/bin/env LD_LIBRARY_PATH=/usr/lib/debug /hurd/foofs

The -P option will make it pause and you will be able to attach ?GDB to the process.

For example

settrans -g -o /hurd/one one.c

gdb one 357

Debugging the hurd with a subhurd

https://www.gnu.org/software/hurd/hurd/debugging/translator.html https://www.gnu.org/software/hurd/hurd/debugging/subhurd.html https://www.gnu.org/software/hurd/hurd/subhurd.html

how to provide a backtrace

debugging glibc changes

hardward watchpoint

you can also use crash server

what is gsync?

building the Hurd try: apt-get build-dep hurd

testing lwip

git send email not supported on the Hurd

poweroff commands. I cannot use reboot. I need to use reboot-hurd or poweroff-hurd to sync discs

subhurds https://www.gnu.org/software/hurd/gdb/backtrace.html https://www.gnu.org/software/hurd/hurd/debugging/glibc.html https://www.gnu.org/software/hurd/hurd/debugging/hardware_watchpoint.html hurd-2017-02-28:21:25:56< teythoon> and the same for gnumach if you want to build that hurd-2017-02-28-21:26:01< teythoon> then clone the repository hurd-2017-02-28-21:26:08< teythoon> autoreconf -fi hurd-2017-02-28-21:26:22< teythoon> the usual ./configure, make dance hurd-2017-02-28-21:26:46< teythoon> building it shouldn't be too problematic hurd-2017-02-28-21:27:02< teythoon> building something that boots is a bit more tricky hurd-2017-02-28-21:27:08< teythoon> good luck ;) hurd-2017-06-03:15:26:28< teythoon> teythoon@hurdbox ~/build/hurd-upstream % remap /servers/socket/2 /tmp/t -- /usr/bin/curl http://192.168.122.1:8123/t hurd-2017-08-21-15:07:04< jlledom> Another question: how do you send mails with git? git send-email is not supported in the hurd, maybe an old git version hurd-2017-11-12-11:23:58< youpi> alternatively, you can use reboot-hurd and poweroff-hurd hurd-2017-11-12-11:54:06< z3ntu> youpi: in a fresh install when running "poweroff-hurd" I get "In tight loop: hit ctrl-alt-del to reboot"

I can actually change the line "In tight loop: hit ctrl-alt-del to reboot" message to "you can now safely turn your computer off"

a command can run as nobody

basic Hurd example of mach IPC with one process

Mach

ports, port rights, and port names

Awesome that is a non-technical contribution that I can do! A process has can have very limited permissions. A nobody process can only do the things you tell it to. http://hurdextras.nongnu.org/ipc_guide/mach_ipc_basic_concepts.html#1.1.1 A port is a message que. You can think of a port as a small box full of envelopes.

A port set is a collection of message ques. You can think of a port set as a desk that has many boxes on top of it. A port set is useful in that a desk is useful in organizing many messages.

functions

paying interface

a task is a thing that a micokernel needs to do

the things that Mach provides that makes it NOT a true nanokernel

The port name is the name of the port. mach_port_t ie: Benjamin. It's just the name of the box of messages. The port name points to an item in the tasks port name space. What's the tasks port name space? The item has either a dead name, a send-once right, or a combination of a receive right and a send right. http://lists.gnu.org/archive/html/l4-hurd/2002-06/msg00001.html

http://darnassus.sceen.net/~hurd-web/microkernel/mach/concepts/

Mach is a first-generation microkernel.

Mach's basic abstractions include virtual address spaces in the form of tasks, execution contexts in the form of threads, IPC, capabilities in the form of ports, and memory objects, which enable Mach's external pager mechanism.

Controlling tasks, their virtual address space, threads, and other system objects in Mach is implemented by using ports, as opposed to other kernels' system call interface: almost all of the Mach API is implemented by sending messages to ports. Device drivers that reside in kernel space are controlled by ports, too.

Mach's ?API is well-documented.

In particular the [mach_kernel_principles] book further elaborates on Mach's concepts and principles. IRC, freenode, #hurd, 2013-08-26

may add messages without checking for integer overflows

inline function are better than macros

gnumach regarding return values

< stargater> then is mach not more microkernel < stargater> when it have driver inside < braunr> mach is a hybrid < braunr> even without drivers < stargater> in www i read mach is microkernel < stargater> not hybrid < braunr> the word microkernel usually includes hybrids < braunr> true microkernels are also called nanokernels < braunr> the word isn't that important, what matters is that mach does more in kernel than what the microkernel principle implies *< braunr> e.g. high level async IPC and high level virtual memory operations* *< braunr> including physical memory management* hurd-2016-06-08-16:15:17< braunr> i just noticed that mach adds the overhead of messages without checking for integer overflows hurd-2017-01-19-16:15:46< braunr> avarzille: you should take the habit of using inline functions instead of macros as often as you can hurd-2017-01-19-16:18:51< avarzille> I'm asking specifically for the sleeping lock hurd-2017-01-19-16:18:58< avarzille> I use 0 (success) and -1 (error) hurd-2017-01-19-16:21:51< braunr> kern_return_t hurd-2017-01-19-16:22:56< avarzille> oh, I thought that was for communicating with userland hurd-2017-01-19-16:23:20< braunr> i'd advise never using a boolean code, unless the function clearly returns a boolean, i.e. true or false hurd-2017-01-19-16:23:31< braunr> no, it's also internal hurd-2017-01-19-16:23:59< avarzille> ok hurd-2017-01-19-16:24:16< braunr> the most common return types are void (of course), int (error code, here kern_return_t), bool (for condition tests), and void * (NULL meaning only possible error such as ENOMEM) hurd-2017-01-19-16:24:21< braunr> people expect that hurd-2017-01-19-16:24:30< braunr> using boolean for error codes is bound to cause confusion

a new strategy of OS design

the two servers in the Hurd that a special

The auth server. This is the server that all programs trust to establish the identity of other servers.

translators

The process server controls system components and does global booking operations. Thu Hurd uses mach ports to communicate info from users and servers.

So for example, when you open a file, the server owing that file node opens a port. The user can then read the file by communicating to the port. When a user normally opens the file, the filesystem opens a port to the file. A filesystem would then jungle a bunch of ports. For every open file, the filesystem would have an open port.

However a file could have a translator associated with it. When a user accesses the file, the filesystem server then runs the translator program. That translator program receives a port to the contents of the file, and returns a port to the user. The translator then translates the underlying information in a different way.

But when a different user accesses that file, they see the original file. Not the translator.

The xmlfs translator does this. It translates an xml file into a directory tree. Then one can cd into what used to be an xml file. How cool is that!?

IPC

A translator could be used to untar a tar file. This would perhaps be a bit slower, but also potentially convenient. Mach's IPC allows syncronous and asyncronous communication.

mach_msg vs. mach_msg_trap

It also allows the transmission of VERY large data.

mach_msg_trap I believe is used when userspace stuff wants to send messages.

mach's vm vs. radixvm ?

mach now uses x15's physical memory management? irc log from 2015

gnu mach uses a slab allocater and an rbtree from x15 radix trees replaced ipc tables as well

RPC types should be fixed sized?

vm_page_allocator has been merged into gnumach from x15

has general locks, braunr wants to replace them with

mach_msg is the kernel one...Both are essentially syscalls. hurd-2015-04-17:13:44:25< teythoon> i have integrated x15's physical memory management into gnumach and got it to compile while still using the old routines hurd-2016-09-26:21:42:38< braunr> since early this year, i merged the vm_page allocator from x15 into gnumach hurd-2016-11-02-14:47:54< braunr> and i'd like to see them gone and replaced with two new types, close to the posix ones hurd-2016-11-02-14:47:58< braunr> "mutex" and "rwlock" hurd-2016-11-02-14:48:26< braunr> the optimization i'm thinking about here is described in "futexes are tricky"

Mach has the ability to create mach servers. The Hurd doesn't want to use these things!

also youpi is an expert on concurrency

a big design mistake

There was some support for drivers in the kernel via mach servers, but this will hopefully be abandoned one day.

genode and l4

x15

GNUMach only has async IPC. L4 only has sync IPC. Clearly sync IPC was the better decision. https://www.youtube.com/user/drsartakov?feature=watch

X15 is Richard Braun's micorkernel. Richard Braun is a veteran Hurd developer, who still occasionally chips into to help develop from time to time. He believes that a total rewrite of the Hurd is the best way forward. X15 is a new microkernel that reuses many ideas from Mach, while improving on others. The Hurd developers occasionally can occasionally use some code from x15 inside GNU Mach, but x15 does not intend to replace GNU Mach.

rdxtrees are used with mutexes...

vm_page vs. mach's vm_page

The Hurd's vm_page uses a buddy allocator from x15 instead of a single list...Though I believe the buddy allocator was ported to gnumach.

#x15 has its own irc channel

as of 2015 x15 does not have ipc

braunr wants true unix signals

a slab allocator is used for kernel memory allocations... hurd-2016-12-23-20:03:17< braunr> but i also want true unix signals hurd-2016-12-23-20:03:26< braunr> well, true async userspace interrupts

s16 is another microkernel?

hurd-2015-09-21-15:48:45< JX7P> this is why i wonder whether some of you folk may find S16 interesting hurd-2015-09-21-15:49:02< braunr> but we could have something like singularity, no userspace, and we'd be fine as long as the interfaces and the isolation are actually there hurd-2015-09-21-15:49:12< braunr> JX7P: definitely hurd-2015-09-21:15:49:35< braunr> (i also find it interesting you named it s16, considering my own project is x15, but that's just a fun fact to notice)

IDL

IDL stands for an interface definition language, and it is used to set up communication between servers and clients to facilitate remote procedure calls.

TODO web website [1/7]

TODO add to translator primer [2/3]

DONE I can mention settrans ~/.signature /hurd/run /usr/games/fortune

DONE I need to change the number 3 to 4 on the page.

TODO I can mention more indepth pages at the bottom I did this?

State "TODO" from [2017-04-27 Thu 16:39]
State "TODO" from [2017-04-27 Thu 16:13]
State "DONE" from "TODO" [2017-04-27 Thu 16:39]
State "TODO" from [2017-04-27 Thu 16:13]
State "DONE" from "TODO" [2017-05-09 Tue 12:50]
State "TODO" from [2017-04-27 Thu 16:44]
State "TODO" from [2017-04-27 Thu 16:14]

TODO debugging subhurd I did this?

State "TODO" from [2017-04-27 Thu 16:29]

TODO hurd/translator.html I did this?

Any documentation online that says "kvm" let it be known that qemu-system-i386 --enable-kvm is the same thing.

TODO debugging/rptrace.html

State "TODO" from [2017-04-27 Thu 16:31]
State "TODO" from [2017-04-27 Thu 16:34]

TODO mach print syscall

State "TODO" from [2017-04-27 Thu 16:36]

TODO the debugging page doesn't have a portinfo specific page

DONE explain how to check for translator options

State "TODO" from [2017-04-27 Thu 16:39]
State "DONE" from "TODO" [2017-05-09 Tue 13:13]
State "TODO" from [2017-05-06 Sat 18:10]


/hurd/hello --help

Usage: hello [OPTION...] A translator providing a warm greeting.

-c, --contents=STRING Specify the contents of the virtual file -?, --help Give this help list --usage Give a short usage message -V, --version Print program version

Mandatory or optional arguments to long options are also mandatory or optional for any corresponding short options.

Report bugs to .

update hardware compatibility list

remove references of upstart

file:///home/joshua/programming/gnu/hurd/vm/web-page/home/joshua/programming/web.rendered/microkernel/mach/gnumach/hardware_compatibility_list.html

new driver theory talks about DDE but not rump

physical memory management page says that the Hurd's max memory is 800MB, but this is not true anymore.

hurdish TCP/IP google summer of code stack not uses lwip not pfinit. can we update this page

gsoc sound support pulseaudio works on the hurd now...

remove Add UTIME_NOW and UTIME_OMIT. Samuel apparently is about to finish this.

microkernel

what makes a microkernel fast?

multiserver microkernelt

Upstart is not maintained. https://www.gnu.org/software/hurd/community/gsoc/project_ideas/driver_glue_code.html https://www.gnu.org/software/hurd/community/gsoc/project_ideas/physical_memory_management.html https://www.gnu.org/software/hurd/community/gsoc/project_ideas/tcp_ip_stack.html https://www.gnu.org/software/hurd/community/gsoc/project_ideas/sound.html https://www.gnu.org/software/hurd/contributing.html#index1h2 small size? http://darnassus.sceen.net/~hurd-web/microkernel/fud/ l4 different schedulers in userspace. The microkernel only implements basic memory management, IPC.

Even the scheduler could run in userspace! That's what l4 does. This means that different processes could run on different schedulers!

The rest of the traditional kernel, TCP/IP, filesystems, device drivers, can all run in userspace as separate processes. This means a normal user can modify things that would normally be in the kernel.

AHCI is normally needed for multicpu support? but it is not for x15? acpica does this in x15?

Debian GNU/Hurd

use "su -" for programs in /sbin instead of "su"

setting up the default debian environment

It's also possible to have a microkernel running, and then run one single userspace process for TCP/IP, filesystems, device drivers, etc. That's essentially what mac does... I think. First, thanks for informing about "su -" needed for /sbin commands to be used rather than just "su", indeed it fixed my problem.

if you cannot click to open a default terminal:

So after installing the daily image,tasksel, selected "Debian Desktop environment", "dpkg-reconfigure x11-common xserver-xorg-legacy" as described in https://www.debian.org/ports/hurd/hurd-install , startx, left-clicking lower-right icon of a folder, choosing "Open in Terminal" in the menu, I get a popup window that say: Failed to execute child process "bash": Failed to fdwalk: Too many open files in the system.

I have xterm (and lxterm) working in Xorg now, but I did not figure out why... yet.

First, trying to do a program with fdwalk, I have come to the conclusion fdwalk does not exist in Debian sid.

My program:

paul@debian:~$ cat fdwalk.c #include #include #include

struct rlimit rl; int i; int num_files=0; int fd;

int countOpenFiles(void *data, int fd){ num_files++; }

void main(){ fdwalk(countOpenFiles, &fd); printf("num_files=%d",num_files); }

result in:

paul@debian:~$ gcc fdwalk.c fdwalk.c: In function ‘main’: fdwalk.c:15:3: warning: implicit declaration of function ‘fdwalk’ [-Wimplicit-function-declaration] 15 | fdwalk(countOpenFiles, &fd); | ^~~~~~ /usr/bin/ld: /tmp/cc3BVAdS.o: in function `main': fdwalk.c:(.text+0x4b): undefined reference to `fdwalk' collect2: error: ld returned 1 exit status paul@debian:~$

also: paul@debian:~$ man fdwalk No manual entry for fdwalk

After that I did: apt-get build-essential; apt-get source bash; apt-get build-dep bash

And I tried to find fdwalk in bahs-5.1 directory with: paul@debian:~$ cd bash-5.1/ paul@debian:~/bash-5.1$ grep -rnw '/path/to/somewhere/' -e 'fdwalk' grep: /path/to/somewhere/: No such file or directory paul@debian:~/bash-5.1$ grep -rnw . -e 'fdwalk' paul@debian:~/bash-5.1$ grep -rn . -e 'fdwalk' paul@debian:~/bash-5.1$ [I did try with other terms found in files, and that worked]

But I decided to rebuild it with: dpkg-buildpackage then installed generated pkgs with dpkg -i [most .deb except static one] in ..

apt-get install xterm twm

after that I created: paul@debian:~$ cat .xinitrc xterm & exec twm

paul@debian: startx

And then, I was able to position my xterm window on the screen and use it.

looking for programs in debian via codesearch

I don't know yet if it is the fact to build my own .xinitrc file that did the trick or to rebuild the bash package... will probably reinstall and try just the .xinitrc file with "apt install xterm twm" to see if it is enough to make it work.

https://codesearch.debian.net/search?q=fdwalk&hl=en

fixing various installer problems in a debian environment

I guess it's coming from libreadline.

Trying to summarize what I have found.

Default X resolution with QEMU is too big, I set it to the minimum size: 1024x768 with: root@debian:/home/paul# cat /etc/X11/xorg.conf.d/paul.conf Section "Device" Identifier "mydev" EndSection

Section "Monitor" Identifier "mymon" EndSection

Section "Screen" Identifier "myscr" Monitor "mymon" Device "mydev" DefaultDepth 24 SubSection "Display" Depth 24 Modes "1024x768" EndSubSection EndSection root@debian:/home/paul#

When using xterm inside .xinitrc, it works. But starting any shell with twm's menu, it open a windows with the error: Failed to fdwalk: Too many open files in the system

Here is my .initrc file: root@debian:/home/paul# cat .xinitrc xrdb -load $HOME/.Xresources & xsetroot -solid gray &

exec twm root@debian:/home/paul#

The cursor tend to go away... I can see some artifacts when moving the mouse... and this why I commented out xterm line, seems the less programs I have, the more the cursor stay visible... Without my own .xinitrc, I sometimes loose the cursor even on the host, strangely.

Also, I do "startx" to start Xorg (with twm)... when I do Ctrl-C, I have to "su", "ps a", and "kill -9 [pid of Xorg]" as the normal user, does not seems able to stop the Xorg instance launched as root.

Looking at: https://sources.debian.org/src/glib2.0/2.66.4-1/glib/gspawn.c/?hl=1213#L1213 I can see that fdwalk seems to have a fallback implementation there, when it is not on the system. I suppose there is an equivalent for Debian/Hurd on: http://git.savannah.gnu.org/cgit/hurd/glibc.git/ but I did not explore much yet.

That's it for now.

things I don't understand

pflocal?

file_name_lookup is related?? Hurd hacking guide 29

w/o this pipe 2 doesn't work

man 2 pipe

/servers/socket/1 ?

/servers/socket/2 is for the IPv4 stack

/servers/socket/26 is for IPv6

/dev/eth0 is DDE

accessing pci cards as user

fsysopts /servers/pci --uid 1234 --p 00:1f.3

Allows a user to access a pci card?

/hurd/hostmux?

Or I can configure it permanently via settrans

I believe this lets you mutate the host name.

so you would do a magical incantation like (this might be wrong):

settrans -c ftp: /hurd/hostmux /hurd/ftpfs

Now you can do something like:

emacs ftp://gnu.org/2023/emacs/README

And it will just work!

remap

remap /bin/sh $HOME/bin/sh

Now my scripts will remap /bin/sh to ~/bin/sh

remap /bin $HOME/unionbin

settrans -fga $HOME/servers/socket/2 /hurd/lwip -6 $HOME/servers/socket/26 -i $HOME/dev/tun0 -a 10.8.0.6 -m 255.255.255.0 -g 10.8.0.1

/bin will now look in ~/unionbin

What does this do?

what is and why should it be ported?

Things I could potentially write for

the hurd

ext3fs

http://darnassus.sceen.net/~jlledom/en/09-point-to-point.html http://savannah.gnu.org/task/?5498

libcap providing capabilities on the hurd

I've downloaded the patch to support ext3fs: https://www.gnu.org/software/hurd/community/gsoc/project_ideas/libcap.html

nfs and nfsd

finish the valgrind port

bcachefs

I could write a dummy library first. https://www.gnu.org/software/hurd/community/gsoc/project_ideas/valgrind.html https://www.patreon.com/posts/status-update-11721750

xmlfs doesn't update the directory structure when the file is saved.

90% of the code runs in userspace! That means that bcachefs could be ported to the Hurd with little difficulty.

But it does work!

settrans -ca xml /hurd/xmlfs example.xml

firewall translator

Make the Hurd console's configuration use xkb layout/variant instead of keymap.

multiprocessor support

random translator isn't this done?

overwriting default servers

gtk+ driver

copy and paste support for console

ftpfs hangs

string_t is limited to 1024 bytes

make a /proc/cpuinfo

test driver development

run glib's and other software's test-suites

pfinet

rewrite pfinet

pfinet hangs

http://savannah.gnu.org/task/?12723 http://xkbcommon.org/doc/current/index.html could be used for On the contributing page https://www.gnu.org/software/hurd/contributing.html http://savannah.gnu.org/task/?7179 http://savannah.gnu.org/task/?5130 http://savannah.gnu.org/task/?6612 http://savannah.gnu.org/task/?1427 http://savannah.gnu.org/bugs/?24383 http://savannah.gnu.org/bugs/?28511 https://www.gnu.org/software/hurd/community/gsoc/project_ideas/testing_framework.html https://www.gnu.org/software/hurd/community/gsoc/project_ideas/testsuites.html http://savannah.gnu.org/task/?5469 There is a google summer of project for 2017 to do this. http://savannah.gnu.org/bugs/?27539

NSF is pretty much crap

porting programs to the Hurd that fail because of PATH_MAX

http://savannah.gnu.org/task/?5497

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/maxpath.html

currently abiword isn't working on the hurd because of this.

trying to extend glibc for the Hurd

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/testsuites.html

https://www.gnu.org/software/hurd/source_repositories/glibc.html

https://www.gnu.org/software/hurd/open_issues/glibc.html#index4h2

https://www.gnu.org/software/hurd/glibc.html

https://www.gnu.org/software/hurd/open_issues/glibc.html

https://www.gnu.org/software/hurd/glibc/mmap.html

modify programs to use the hurd translaters

automake'ify hurd's code base

add an input driver for console client

moving mount/umount into libc is a easy hacking thing to do apparently. https://www.gnu.org/software/hurd/community/gsoc/project_ideas/download_backends.html http://savannah.gnu.org/task/?5491 http://savannah.gnu.org/task/?2503

documentation things I can fix

Hurd hacking guide

page 34

Why should I learn about Mach if the Hurd will switch to L4 soon?

hurd.texi

TODO "3.4 Misc Library" There is a hurd library called "libshouldbeinlibc". Some of those functions are added into the glibc.

TODO 5.1 Translators FIXME We need a good example

TODO 5.1.3 Invoking mount

TODO 5.4.8 Notifications has nothing in it

TODO 5.4.9 File Translators Fixme how to set and get translators

TODO 5.5 Filesystem interface

TODO 8 Stored Filesystems has almost no information

TODO 8.2 Linux Extended 2 FS

TODO 9 Twisted Filesystems

TODO 10 Distributed Filesystems

TODO 11 Networking

TODO 12 Terminal handling

TODO 13 Has no information

TODO 14 has no information

add in a section to hurd.texi about running the Hurd in qemu

things I finished today

State "TODO" from [2016-06-02 Thu 07:18]
State "TODO" from [2016-06-02 Thu 07:25]
State "TODO" from [2016-06-02 Thu 07:37]
State "TODO" from [2016-06-02 Thu 07:44]
State "TODO" from [2016-06-02 Thu 07:45]
State "TODO" from [2016-06-02 Thu 07:45]
State "TODO" from [2016-06-02 Thu 07:47]
State "TODO" from [2016-06-02 Thu 07:48]
State "TODO" from [2016-06-02 Thu 07:49]
State "TODO" from [2016-06-02 Thu 07:50]
State "TODO" from [2016-06-02 Thu 07:54]
State "TODO" from [2016-06-02 Thu 07:55]
State "TODO" from [2016-06-02 Thu 07:55]
State "TODO" from [2016-06-02 Thu 07:55]

DONE "3.1 Threads Library" claims that the Hurd uses cthreads, when it actually uses pthreads

DONE it mentions libtreefs should be talked about in the

State "DONE" from "TODO" [2017-04-22 Sat 17:20]
State "TODO" from "TODO" [2017-04-22 Sat 17:19]
State "TODO" from [2016-06-02 Thu 07:07]
State "DONE" from "TODO" [2017-04-22 Sat 17:23]
State "TODO" from [2017-04-22 Sat 17:23]

DONE added a reference to

DONE I added some detail about how to use the program. I just copied the output of mount --help.

gnumach

[#A] readahead

[#A] multipage paging

let mach create futexes

rewrite mach's IPC and VM

replace many of the simple spin locks with mutexes?

State "DONE" from [2017-04-22 Sat 17:32]
State "DONE" from [2017-04-22 Sat 17:51]

that there are very few reasons that one would use spin locks. These are not very fast.

He wants to replace the "general locks" with something similar to POSIX mutex and rwlock. I can read more about it in the article "futexes are tricky".

gsync_requeue has an implementation & braunr has an implementation as well in x15.

youpi thinks we should use wait queues...

rewrite gnu mach device driver glue code

lots of stuff on this page:

http://savannah.gnu.org/task/?5488

mach 5 IPC communication

clean up the mach code

gnumach uses a lot of spin locks. Brauner thinks they should be replaced with mutexes.

file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/microkernel/mach/gnumach/projects/mach_5.html file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/microkernel/mach/gnumach/projects/clean_up_the_code.html Then mach should provide futexes.

porting packages

bug notes

[#B] mach

mach compile errors

linux/dev/init/main.c: In function 'alloc_contig_mem':

linux/dev/glue/block.c: In function 'device_open':

linux/dev/glue/block.c: In function 'device_get_status':

linux/dev/drivers/block/floppy.c: In function 'fd_ioctl':

In file included from linux/dev/drivers/block/ahci.c:32:0:

linux/src/drivers/block/triton.c: In function 'config_drive_for_dma':

In file included from ./linux/src/include/linux/string.h:47:0,

linux/src/drivers/block/triton.c:253:24: warning: pointer targets in passing arg

In file included from ./linux/src/include/linux/string.h:47:0,

In file included from linux/src/drivers/scsi/hosts.c:129:0:

In file included from linux/src/drivers/scsi/hosts.c:141:0:

quite a few more redefined defs

In file included from linux/src/drivers/scsi/hosts.c:117:0:

linux/src/drivers/scsi/scsi.c: In function 'scan_scsis':

linux/src/drivers/scsi/scsi.c:95:12: note: expected 'char *' but argument is of

linux/src/drivers/scsi/scsi.c:491:9: warning: pointer targets in passing argumen

linux/src/drivers/scsi/scsi.c:95:12: note: expected 'char *' but argument is of

linux/src/drivers/scsi/scsi.c:525:23: warning: pointer targets in passing argume

In file included from linux/src/drivers/scsi/scsi.c:57:0:

linux/src/drivers/scsi/scsi.c: In function 'scan_scsis_single': LOTS HERE

http://lists.gnu.org/archive/html/bug-hurd/2012-12/msg00002.html file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/community/weblogs/ArneBab/porting-simple-packages.html linux/dev/init/main.c:299:3: warning: return makes pointer from integer without a cast return phystokv(m); ^ linux/dev/init/main.c:196:15: warning: 'addr' may be used uninitialized in this function [-Wmaybe-uninitialized] vm_offset_t addr, max_addr; linux/dev/glue/block.c:1160:14: warning: assignment from incompatible pointer ty pe devp = IP_NULL; ^ linux/dev/glue/block.c:1669:28: warning: initialization from incompatible pointe r type struct disk_parms *dp = status; linux/dev/drivers/block/floppy.c:3314:32: warning: pointer targets in passing ar gument 1 of 'normalize_0x02xx_ioctl' differ in signedness [-Wpointer-sign] ECALL(normalize_0x02xx_ioctl(&cmd, &size)); ./linux/dev/include/linux/blk.h:167:21: warning: 'do_sd' defined but not used [- Wunused-variable] #define DEVICE_INTR do_sd linux/src/drivers/block/triton.c:223:24: warning: pointer targets in passing arg ument 2 of 'strcmp' differ in signedness [-Wpointer-sign] if (!strcmp(*list++,id->model)) { from ./linux/src/include/asm/termios.h:59, from ./linux/src/include/linux/termios.h:5, from ./linux/src/include/linux/tty.h:20, from ./linux/dev/include/linux/sched.h:26, from ./linux/dev/include/linux/mm.h:4, from linux/src/drivers/block/triton.c:22: ./linux/dev/include/asm/string.h:108:19: note: expected 'const char *' but argum ent is of type 'unsigned char *' static inline int strcmp(const char cs,const char * ct) ument 2 of 'strcmp' differ in signedness [-Wpointer-sign] if (!strcmp(list++,id->model)) { ^ from ./linux/src/include/asm/termios.h:59, from ./linux/src/include/linux/termios.h:5, from ./linux/src/include/linux/tty.h:20, from ./linux/dev/include/linux/sched.h:26, from ./linux/dev/include/linux/mm.h:4, from linux/src/drivers/block/triton.c:22: ./linux/dev/include/asm/string.h:108:19: note: expected 'const char *' but argum ent is of type 'unsigned char *' static inline int strcmp(const char cs,const char * ct) linux/src/drivers/scsi/dtc.h:49:0: warning: "CSR_RESET" redefined #define CSR_RESET 0x80 /* wo Resets 53c400 / ^ In file included from linux/src/drivers/scsi/hosts.c:105:0: linux/src/drivers/scsi/in2000.h:187:0: note: this is the location of the previou s definition #define CSR_RESET 0x00 linux/src/drivers/scsi/ncr53c8xx.h:129:0: warning: "SCSI_NCR_SETUP_ULTRA_SCSI" r edefined #define SCSI_NCR_SETUP_ULTRA_SCSI (2) ^ In file included from linux/src/drivers/scsi/sym53c8xx.h:60:0, from linux/src/drivers/scsi/hosts.c:137: linux/src/drivers/scsi/sym53c8xx_defs.h:154:0: note: this is the location of the previous definition #define SCSI_NCR_SETUP_ULTRA_SCSI (3) linux/src/drivers/scsi/ncr53c8xx.h:129:0: warning: "SCSI_NCR_SETUP_ULTRA_SCSI" r edefined #define SCSI_NCR_SETUP_ULTRA_SCSI (2) ^ In file included from linux/src/drivers/scsi/sym53c8xx.h:60:0, from linux/src/drivers/scsi/hosts.c:137: linux/src/drivers/scsi/sym53c8xx_defs.h:154:0: note: this is the location of the previous definition #define SCSI_NCR_SETUP_ULTRA_SCSI (3) linux/src/drivers/scsi/qlogicisp.h:72:30: warning: 'proc_scsi_isp1020' defined b ut not used [-Wunused-variable] static struct proc_dir_entry proc_scsi_isp1020; linux/src/drivers/scsi/scsi.c:454:32: warning: pointer targets in passing argume nt 9 of 'scan_scsis_single' differ in signedness [-Wpointer-sign] &SDpnt, SCpnt, shpnt, scsi_result); type 'unsigned char *' static int scan_scsis_single (int channel,int dev,int lun,int max_scsi_dev , t 9 of 'scan_scsis_single' differ in signedness [-Wpointer-sign] scsi_result) type 'unsigned char ' static int scan_scsis_single (int channel,int dev,int lun,int max_scsi_dev , nt 1 of 'scsi_init_free' differ in signedness [-Wpointer-sign] scsi_init_free (scsi_result, 512); linux/src/drivers/scsi/hosts.h:327:13: note: expected 'char ' but argument is o f type 'unsigned char *' extern void scsi_init_free(char ptr, unsigned int size); linux/src/drivers/scsi/scsi.c:708:18: warning: pointer targets in passing argume nt 1 of 'print_inquiry' differ in signedness [-Wpointer-sign] print_inquiry (scsi_result); ^ linux/src/drivers/scsi/scsi.c:93:13: note: expected 'unsigned char ' but argume nt is of type 'char *' static void print_inquiry(unsigned char *data); ^ linux/src/drivers/scsi/scsi.c:731:30: warning: pointer targets in passing argume nt 1 of 'get_device_flags' differ in signedness [-Wpointer-sign] bflags = get_device_flags (scsi_result); ^ linux/src/drivers/scsi/scsi.c:303:12: note: expected 'unsigned char *' but argum ent is of type 'char *' static int get_device_flags(unsigned char response_data){ ^ linux/src/drivers/scsi/scsi.c: In function 'resize_dma_pool': linux/src/drivers/scsi/scsi.c:2898:21: warning: pointer targets in passing argum ent 1 of 'scsi_init_free' differ in signedness [-Wpointer-sign] scsi_init_free(dma_malloc_pages[i], PAGE_SIZE);

linux/src/drivers/scsi/scsi.c:2513:6: note: expected 'char ' but argument is of type 'unsigned char *' void scsi_init_free(char ptr, unsigned int size)

linux/src/drivers/scsi/scsicam.c: In function 'scsicam_bios_param':

2 linux/src/drivers/scsi/sd.c: In function 'revalidate_scsidisk':

linux/src/drivers/scsi/sr.c: In function 'sr_detach':

6 linux/src/drivers/scsi/sr_ioctl.c: In function 'sr_ioctl':

linux/src/drivers/scsi/scsicam.c:58:5: warning: implicit declaration of function 'scsi_partsize' [-Wimplicit-function-declaration] ret_code = scsi_partsize (bh, (unsigned long) size, (unsigned int *) ip + 2 , ^ linux/src/drivers/scsi/sd.c:1551:9: warning: unused variable 'devi' [-Wunused-va riable] kdev_t devi = MKDEV(MAJOR_NR, minor); ^ linux/src/drivers/scsi/sd.c: In function 'sd_detach': linux/src/drivers/scsi/sd.c:1597:10: warning: unused variable 'devi' [-Wunused-v ariable] kdev_t devi = MKDEV(MAJOR_NR, minor); ^ In file included from ./linux/dev/include/linux/blk.h:6:0, from linux/src/drivers/scsi/sd.c:42: linux/src/drivers/scsi/sr.c:1218:13: warning: unused variable 'devi' [-Wunused-v ariable] kdev_t devi = MKDEV(MAJOR_NR, i); linux/src/drivers/scsi/sr_ioctl.c:219:9: warning: pointer targets in assignment differ in signedness [-Wpointer-sign] buffer = (unsigned char *) scsi_malloc(512); ^ linux/src/drivers/scsi/sr_ioctl.c:361:9: warning: pointer targets in assignment differ in signedness [-Wpointer-sign] buffer = (unsigned char *) scsi_malloc(512); ^ linux/src/drivers/scsi/sr_ioctl.c:377:7: warning: pointer targets in assignment differ in signedness [-Wpointer-sign] mask = (unsigned char *) scsi_malloc(512); ^ linux/src/drivers/scsi/sr_ioctl.c:427:9: warning: pointer targets in assignment differ in signedness [-Wpointer-sign] buffer = (unsigned char *) scsi_malloc(512); ^ linux/src/drivers/scsi/sr_ioctl.c:463:9: warning: pointer targets in assignment differ in signedness [-Wpointer-sign] buffer = (unsigned char*) scsi_malloc(512); ^ linux/src/drivers/scsi/sr_ioctl.c:505:9: warning: pointer targets in assignment differ in signedness [-Wpointer-sign] buffer = (unsigned char*) scsi_malloc(512);

In file included from linux/src/drivers/scsi/AM53C974.c:18:0:

2 linux/src/drivers/scsi/AM53C974.c: In function 'AM53C974_information_transfer':

In file included from linux/src/drivers/scsi/BusLogic.c:53:0:

linux/src/drivers/scsi/AM53C974.c: In function 'AM53C974_sync_neg': linux/src/drivers/scsi/AM53C974.h:291:48: warning: variable 'io_port' set but no t used [-Wunused-but-set-variable] #define AM53C974_local_declare() unsigned long io_port linux/src/drivers/scsi/AM53C974.c:1422:21: warning: pointer targets in assignmen t differ in signedness [-Wpointer-sign] cmd->SCp.ptr = (unsigned char *)cmd->SCp.buffer->address; ^ linux/src/drivers/scsi/AM53C974.c:1398:26: warning: variable 'ret' set but not u sed [-Wunused-but-set-variable] int ret, i, len, residual=-1; linux/src/drivers/scsi/FlashPoint.c: In function 'FlashPoint_AbortCCB': linux/src/drivers/scsi/FlashPoint.c:4509:8: warning: variable 'TID' set but not used [-Wunused-but-set-variable] UCHAR TID; ^ linux/src/drivers/scsi/BusLogic.c: In function 'BusLogic_DriverInfo': linux/src/drivers/scsi/BusLogic.c:176:3: warning: pointer targets in return diff er in signedness [-Wpointer-sign] return HostAdapter->FullModelName; ^ linux/src/drivers/scsi/BusLogic.c: In function 'BusLogic_ReadHostAdapterConfigur ation': linux/src/drivers/scsi/BusLogic.c:1519:14: warning: pointer targets in passing a rgument 1 of 'strcpy' differ in signedness [-Wpointer-sign] strcpy(HostAdapter->FirmwareVersion, FlashPoint_FirmwareVersion);

In file included from ./linux/src/include/linux/string.h:47:0, from ./linux/src/include/asm/termios.h:59, from ./linux/src/include/linux/termios.h:5, from ./linux/src/include/linux/tty.h:20, from ./linux/dev/include/linux/sched.h:26, from ./linux/dev/include/linux/blkdev.h:5, from linux/src/drivers/scsi/BusLogic.c:37: ./linux/dev/include/asm/string.h:31:22: note: expected 'char ' but argument is of type 'unsigned char *' static inline char strcpy(char * dest,const char *src) ^ linux/src/drivers/scsi/BusLogic.c:1600:12: warning: pointer targets in passing a rgument 1 of 'strcpy' differ in signedness [-Wpointer-sign] strcpy(HostAdapterModelNumber, "542B");

In file included from ./linux/src/include/linux/string.h:47:0, from ./linux/src/include/asm/termios.h:59, from ./linux/src/include/linux/termios.h:5, from ./linux/src/include/linux/tty.h:20, from ./linux/dev/include/linux/sched.h:26, from ./linux/dev/include/linux/blkdev.h:5, from linux/src/drivers/scsi/BusLogic.c:37: ./linux/dev/include/asm/string.h:31:22: note: expected 'char ' but argument is of type 'unsigned char *' static inline char strcpy(char * dest,const char *src) ^ linux/src/drivers/scsi/BusLogic.c:1607:12: warning: pointer targets in passing a rgument 1 of 'strcpy' differ in signedness [-Wpointer-sign] strcpy(HostAdapterModelNumber, "742A");

In file included from ./linux/src/include/linux/string.h:47:0, from ./linux/src/include/asm/termios.h:59, from ./linux/src/include/linux/termios.h:5, from ./linux/src/include/linux/tty.h:20, from ./linux/dev/include/linux/sched.h:26, from ./linux/dev/include/linux/blkdev.h:5, from linux/src/drivers/scsi/BusLogic.c:37: ./linux/dev/include/asm/string.h:31:22: note: expected 'char ' but argument is of type 'unsigned char *' static inline char strcpy(char * dest,const char *src) ^ linux/src/drivers/scsi/BusLogic.c:1611:12: warning: pointer targets in passing a rgument 1 of 'strcpy' differ in signedness [-Wpointer-sign] strcpy(HostAdapterModelNumber, "747A");

In file included from ./linux/src/include/linux/string.h:47:0, from ./linux/src/include/asm/termios.h:59, from ./linux/src/include/linux/termios.h:5, from ./linux/src/include/linux/tty.h:20, from ./linux/dev/include/linux/sched.h:26, from ./linux/dev/include/linux/blkdev.h:5, from linux/src/drivers/scsi/BusLogic.c:37: ./linux/dev/include/asm/string.h:31:22: note: expected 'char ' but argument is of type 'unsigned char *' static inline char strcpy(char * dest,const char *src) ^ linux/src/drivers/scsi/BusLogic.c:1665:14: warning: pointer targets in passing a rgument 1 of 'strcmp' differ in signedness [-Wpointer-sign] if (strcmp(HostAdapter->FirmwareVersion, "3.3") >= 0)

In file included from ./linux/src/include/linux/string.h:47:0, from ./linux/src/include/asm/termios.h:59, from ./linux/src/include/linux/termios.h:5, from ./linux/src/include/linux/tty.h:20, from ./linux/dev/include/linux/sched.h:26, from ./linux/dev/include/linux/blkdev.h:5, from linux/src/drivers/scsi/BusLogic.c:37: ./linux/dev/include/asm/string.h:108:19: note: expected 'const char ' but argum ent is of type 'unsigned char *' static inline int strcmp(const char cs,const char * ct) ^ linux/src/drivers/scsi/BusLogic.c:1829:19: warning: pointer targets in passing a rgument 1 of 'strcmp' differ in signedness [-Wpointer-sign] if (strcmp(HostAdapter->ModelName, "BT-757") == 0)

In file included from ./linux/src/include/linux/string.h:47:0, from ./linux/src/include/asm/termios.h:59, from ./linux/src/include/linux/termios.h:5, from ./linux/src/include/linux/tty.h:20, from ./linux/dev/include/linux/sched.h:26, from ./linux/dev/include/linux/blkdev.h:5, from linux/src/drivers/scsi/BusLogic.c:37: ./linux/dev/include/asm/string.h:108:19: note: expected 'const char ' but argum ent is of type 'unsigned char *' static inline int strcmp(const char cs,const char * ct) ^ linux/src/drivers/scsi/BusLogic.c:1873:14: warning: pointer targets in passing a rgument 1 of 'strcmp' differ in signedness [-Wpointer-sign] if (strcmp(HostAdapter->FirmwareVersion, "3.31") >= 0)

In file included from ./linux/src/include/linux/string.h:47:0, from ./linux/src/include/asm/termios.h:59, from ./linux/src/include/linux/termios.h:5, from ./linux/src/include/linux/tty.h:20, from ./linux/dev/include/linux/sched.h:26, from ./linux/dev/include/linux/blkdev.h:5, from linux/src/drivers/scsi/BusLogic.c:37: ./linux/dev/include/asm/string.h:108:19: note: expected 'const char ' but argum ent is of type 'unsigned char *' static inline int strcmp(const char cs,const char * ct) ^ linux/src/drivers/scsi/BusLogic.c:1899:18: warning: pointer targets in passing a rgument 1 of 'strcmp' differ in signedness [-Wpointer-sign] if (strcmp(HostAdapter->FirmwareVersion, "4.22") >= 0) ^

In file included from ./linux/src/include/linux/string.h:47:0, from ./linux/src/include/asm/termios.h:59, from ./linux/src/include/linux/termios.h:5, from ./linux/src/include/linux/tty.h:20, from ./linux/dev/include/linux/sched.h:26, from ./linux/dev/include/linux/blkdev.h:5, from linux/src/drivers/scsi/BusLogic.c:37: ./linux/dev/include/asm/string.h:108:19: note: expected 'const char ' but argum ent is of type 'unsigned char *' static inline int strcmp(const char cs,const char * ct) ^ linux/src/drivers/scsi/BusLogic.c:1903:18: warning: pointer targets in passing a rgument 1 of 'strcmp' differ in signedness [-Wpointer-sign] if (strcmp(HostAdapter->FirmwareVersion, "3.35") >= 0)

In file included from ./linux/src/include/linux/string.h:47:0, from ./linux/src/include/asm/termios.h:59, from ./linux/src/include/linux/termios.h:5, from ./linux/src/include/linux/tty.h:20, from ./linux/dev/include/linux/sched.h:26, from ./linux/dev/include/linux/blkdev.h:5, from linux/src/drivers/scsi/BusLogic.c:37: ./linux/dev/include/asm/string.h:108:19: note: expected 'const char ' but argum ent is of type 'unsigned char *' static inline int strcmp(const char cs,const char * ct) ^ linux/src/drivers/scsi/BusLogic.c:1928:14: warning: pointer targets in passing a rgument 1 of 'strcmp' differ in signedness [-Wpointer-sign] strcmp(HostAdapter->ModelName, "BT-445S") == 0 &&

Gnumach crashes after rpctrace exists unexpectedly

string_t is limited to 1024 bytes

gnumach doesn't check device name

TODO I should add an INSTALL file into gnumach. it'll say

http://savannah.gnu.org/bugs/?15300

autoreconf -fi ./configure make make install

TODO I can improve the mach manual by typing a few lines about Mach's history. That would be helpful. I can probably

copy it from the wiki online.

Actually I cannot copy from that page. That wiki page doesn't have the you can copy me section down toward the bottom of the webpage.

MIG

MiG generated code does not destroy invalid reply DOS attacks

MIG fails to check un-terminated strings

Hurd bugs

TODO httpfs is bonkers with no internet [0/2]

http://savannah.gnu.org/bugs/?48456 http://savannah.gnu.org/bugs/?28446

Does libdiskfs need to fix this?

TODO in ~/ try type "cd gnu TAB"

When you are not connected to the internet, this breaks stuff pretty bad. You have to wait like 30 seconds to a minute for the tab completion to fail.

TODO in ~/ type "ls"

You have to wait 30 seconds or so for the translator to die. httpfs should do some sort of test to see if you have an internet connection.

un-initialized variable in init.c

terminal supports making lowercase letters on keyboards that only support uppercase

cannot replace translator is this really a bug?

http://savannah.gnu.org/bugs/?17129 https://savannah.gnu.org/bugs/?48464


$ ls -ld hop
ls: cannot access 'hop': No such file or directory
$ touch hop
$ settrans hop /hurd/fifo
$ settrans hop /hurd/hello
$ showtrans hop
/hurd/fifo


$ ls -ld hop
ls: cannot access 'hop': No such file or directory
$ touch hop
$ settrans hop /hurd/fifo
$ settrans -gf hop
$ settrans hop /hurd/hello
$ showtrans hop
/hurd/hello

Is this really a bug?

#49730 /proc/PID/environ returns I/O errors on read attempts

http://savannah.gnu.org/bugs/?48438 I'm having problems retrieving process environment from /proc. Something like "cat /proc/936/environ" often (but not always) returns an I/O error, while "msgport --getenv -p 936" works fine. I'm not sure exactly which processes suffer from this, but translators seem particularly vulnerable.

Studying the problem with rpctrace, it seems that there are two different ways to obtain a process's environment. The proc server hold a pointer to the environment array (in the process's address space, not its own), and then vm_read's it to answer proc_procgetenv RPCs. This is how procfs does it, and this is what isn't working.

I've written a short program (attached) to fetch the argument locations and print them. Comparing this output to /proc/PID/maps shows that on the affected processes, the addresses appear sane, but do not correspond to mapped memory locations, thus the I/O errors.

The other way to get a process's environment is to fetch the process's msg port and query it using a msg_get_environment RPC. The msgport program does this, and is able to successfully retrieve environment information.

Related: Do we need two different ways to retrieve a process's environment?

Hurd FIFO discarding options

http://savannah.gnu.org/bugs/?48437


$ ls -ld hop
ls: cannot access 'hop': No such file or directory
$ touch hop
$ settrans hop /hurd/fifo --multiple-readers
$ showtrans hop
/hurd/fifo

The output from showtrans should have been "/hurd/fifo --multiple-readers".

limits are not set to children

Really though?

http://savannah.gnu.org/bugs/?43320


$ ulimit -s
unlimited
$ ulimit -s 1024
$ ulimit -s
1024
$ bash
$ ulimit -s
unlimited

device_clone not called on port closure

ext2fs leaks memory on exec calls

http://savannah.gnu.org/bugs/?30256 http://savannah.gnu.org/bugs/?30096

Braunr may have fixed this bug. I should try taking a look at the code on the website and seeing if running that code does indeed cause a memory leak.

translators output garbage with settrans -P and stopped by gdb

application bugs

gdb breakpoints in threads make SIGTRAP

git commit with Emacs -nw fails

http://savannah.gnu.org/bugs/?29642

http://savannah.gnu.org/bugs/?48864

firm-link look results in memory exhaustion

ftpfs hangs if the network is down

libstore segfaults

time returns bogus time

sudo resets the environmental variables

setauth causes a crash

nice values error. already fixed?

glibc is missing SIGRTMIN and SIGRTMAX

glibc missing TLS

When running 'git commit' with environment variable EDITOR set to 'emacs -nw', emacs works correctly to edit the commit message unless CNTL-Z is used to suspend the session. The shell prompt appears correctly, but when 'fg' is used to resume the git/commit, emacs does not redisplay. CNTL-Z can be used again to get back to the shell, but ultimately the 'git commit' has to be killed (with 'kill'). http://savannah.gnu.org/bugs/?29809 http://savannah.gnu.org/bugs/?24383 A patch claims to fix it. http://savannah.gnu.org/bugs/?20906 http://savannah.gnu.org/bugs/?20683 http://savannah.gnu.org/bugs/?20060 http://savannah.gnu.org/bugs/?19767 http://savannah.gnu.org/bugs/?18883 http://savannah.gnu.org/bugs/?17650 http://savannah.gnu.org/bugs/?17644

hurd/crash --suspend fails

resolution in translator issues

proc leaks memory

verifying passwords is UID based

#6613 Hurd duplicates common GNU programs

dd uses up memory

#6612 over ride default servers

fs crashes a subhurd

It's possible already fixed? http://savannah.gnu.org/bugs/?17319 http://savannah.gnu.org/bugs/?17133 http://savannah.gnu.org/bugs/?15355 http://savannah.gnu.org/bugs/?15322 http://savannah.gnu.org/bugs/?15320 http://savannah.gnu.org/task/?6613 http://savannah.gnu.org/bugs/?15297 http://savannah.gnu.org/task/?6612 http://savannah.gnu.org/bugs/?15317

/hurd/crash --suspend doesn't work. I need a program that crashes...

Can this be reproduced?

Well I have one now.


./crash

Segfault

Debian bugs

small hack entries

But I don't understand if that means the bug is still broken? http://savannah.gnu.org/bugs/?17319 https://bugs.debian.org/cgi-bin/pkgreport.cgi?package=hurd#_0_2_4 https://www.gnu.org/software/hurd/contributing.html#index1h2

Small hack entries

Here is a list of small hacks, which can serve as entries into the Hurd code for people who would like to dive into the code but just lack a "somewhere to begin with".

Add UTIME_NOW and UTIME_OMIT. It is a matter of taking the BSD values, add the #defines to the proper header, and implement the support in *_S_file_utimes functions. See also Debian bug #762677.

Some translators do not support fsysopts, i.e. support for the file_get_fs_options and fsys_set_options RPCs.

Extend device_read/device_write into supporting > 2TiB disk sizes.

Make the Hurd console's configuration use xkb layout/variant instead of keymap.

Add NX protection support to GNU Mach.

Write a partfs translator, to which one gives a disk image, and which exposes the partitions of the disk image, using parted, and the parted-based storeio (settrans -c foos1 /hurd/storeio -T typed part:1:file:/home/samy/tmp/foo). This would be libnetfs-based.

Move the mount/umount logic from utils/{,u}mount.c into glibc.

Fix /proc/self. Look at [glibc]/hurd/lookup-retry.c for how FS RETRY MAGICAL lookups work.

Add a tool to trace system calls, by using gnumach's Syscall-Emulation, see http://www.gnu.org/software/hurd/gnumach-doc/Syscall-Emulation.html

Improve our FUSE library.

Add a relatime or lazytime option to ext2fs.

Fix our symlink translator.

Fix chmod on fifos: mkfifo foo ; sudo chmod g+w foo

Strengthen httpfs: it should append '/' to URL automatically, it should not fallback index.html itself, etc.

Fix O_NOATIME, see https://buildd.debian.org/status/fetch.php?pkg=borgbackup&arch=hurd-i386&ver=1.0.2-1&stamp=1460838100

Emacs bugs

Executing this in emacs causes emacs to crash

probably a lot more small easy issues.

#+begin_src emacs-lisp (use-package package-does-not-exist :ensure t) #+end_src

The hurd in qemu

Installing

Add yourself to the kvm group. This group lets you start a kernel-ized virtual machine. You can check to which groups you belong with:


groups joshua

 http video audio

You can add yourself to the kvm group with:


# gpasswd -a user kvm

manually installing

Don't do this unless you know what you are doing. This method is really HARD.


  $ wget https://people.debian.org/~sthibault/hurd-i386/installer/cdimage/netinst.iso
  $ qemu-img create hd0.img 4G
  $ qemu-system-i386 -net nic,model=ne2k_pci -net user               \
  -m 2G --no-reboot -machine kernel_irqchip=off -cdrom debian-sid-20160824-NETINST-1.iso \
 -enable-kvm -curses -drive cache=writeback,file=hd020160824.img,format=raw

If you do not specify format=raw, then you won't be able to write changes to your image. If you do not specify -curses, then you will run the installer graphically, and it may not let you use the up or down arrow keys. BUT using -curses, it may fail to install correctly...It will fail to install if you specify Dvorak as your keyboard, because the Hurd does not support dvorak.

using the pre-installed image Recommended

Make sure you DO not put your user's home directory in a separate partition that is mounted on /home. The current debian port does not support this.


$ wget http://people.debian.org/~sthibault/hurd-i386/debian-hurd.img.tar.gz
$ tar -xz < debian-hurd.img.tar.gz
$ kvm -m 1G -drive cache=writeback,file=$(echo debian-hurd-*.img)

Then resize the image


qemu-img info debian-hurd-20160824.img

image: debian-hurd-20160824.img
file format: raw
virtual size: 2G (11736711168 bytes)
disk size: 1.2G

Then you can run qemu-img resize file size to change the img size


qemu-img resize debian-hurd-20160824.img +5G

Then you need to resize the partitions in the disk image after you have booted into it.

Things to do after installing the hurd

add a new user

set up sudo

~useradd username~ It would be nice to get your normal user to be able to install packages. To do that:


adduser joshua sudo

Alternatively, do it the long way.


$ apt-get install sudo
$ export EDITOR=nano
# visudo

You'll want to add yourself the ability to run sudo, so add this line ~joshua ALL=(ALL) ALL~

Enable automatic filesystem fscking of the / partition in

Control-X will let you close the file. Press "y" to save the file.

/etc/default/rcS

setting up your instant development environment


apt-get update
apt-get install -y git mercurial emacs vim
apt-get build-dep -y hurd gnumach
git clone git://git.sv.gnu.org/hurd/hurd.git
git clone git://git.sv.gnu.org/hurd/gnumach.git
git clone git://git.sv.gnu.org/hurd/incubator.git

connecting to the hurd via ssh

This means that you can run the Hurd in qemu, and if it is super slow, I can develop for the Hurd in GNU/Linux!

Running

regularly running


qemu-system-i386 -m 2G -net nic -net user,hostfwd=tcp::5555-:22 \
-drive file=hd0.img,cache=writeback,format=raw                  \
-curses -enable-kvm                                             \
-no-reboot

The -curses option runs the Hurd in a text layout. It keeps your current keyboard layout. Since I am using Dvorak, I need to pass this option. The Hurd does not yet support dvorak.

The -enable-kvm option tries to run the Hurd as fast as possible by letting the virtual machine run as fast as possible on the CPU.

The -no-reboot option I found info here.

For shutting down, use reboot, then press c in grub and issue halt (to avoid filesystem corruption). Adding --no-reboot to the qemu line should help, too.

connecting to the hurd via ssh

When you see graphic mode, press enter a couple of times It is often convenient to connect through ssh to the box, instead of logging on the console, this can be done by forwarding the ssh port:

$ kvm -net user,hostfwd=tcp:127.0.0.1:2222-:22 -net nic -drive file=debian-hurd*.img,cache=writeback -m 1G

and then you can connect through ssh:

ssh user@localhost -p 2222

In Emacs you can do this with M-x find-file RET /METHOD:USER@HOST#PORT:FILENAME

aka /ssh:joshua@localhost#2222

if you ssh into a new qemu install of the Hurd, the old ssh won't work. You'll have to remove the localhost line in


cat .ssh/known_hosts

ssh.phx.nearlyfreespeech.net,208.94.116.211 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEkD1YQOPCy0w/g/OqpBys337RAtyK/HDmW2i0BUW4mnN5em2jkB69nhJP91nU8s05OfB1MgxVVzyXc80Ounbzw=[localhost]:2222 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN08IEOxaW//sIczLWdLxX2tqD5XCw6WyGapkRKXnx46by/MNADwrz2orWELkmyS6zz03YfeIrtEFQ7TyxF3IFw=

Transferring files from host to qemu

Mounting Disk Image on Host

You may wish to mount your disk image on your host system to transfer files. To do this you will first need to find the offset of the partition you wish to mount.


 fdisk -l hd0.img

Disk hd0.img: 4 GiB, 4294967296 bytes, 8388608 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0xb5b4d7a0

Device Boot Start End Sectors Size Id Type hd0.img1 2048 7938047 7936000 3.8G 83 Linux hd0.img2 7940094 8386559 446466 218M 5 Extended hd0.img5 7940096 8386559 446464 218M 82 Linux swap / Solaris

Now take the number of sectors for the beginning of the partition and multiply it by the sector size. My / partition starts at sector 2048 and I have a sector size of 512 therefore my offset is 1048576.

The partition starts at 1955840. What's my sector size?


(* 1955840 512)

1001390080

Then you mount the device on a directory! Not a file.

The actual command that I use is:


sudo mount -o loop,offset=1001390080 hd020160824.img ./web-page

trouble booting the Hurd

youpi: depending where it hangs, perhaps disabling the hur dconsole would help, by setting ENABLE to false in /etc/default/hurd-console

How to start the Hurd console from the mach console (info here):

# console -d vga -d pc_mouse --repeat=mouse -d pc_kbd --repeat=kbd -d generic_speaker -c /dev/vcs

updating (on debian systems)

sudo apt-get update # Fetches the list of available updates sudo apt-get upgrade # Strictly upgrades the current packages sudo apt-get dist-upgrade # Installs updates (new ones)

hurd qemu notes

apt-get autoremove #removes old packages This lets you write stuff to disk qemu-system-i386 -drive file=debian-hurd-20150320.img,cache=writeback,format=raw

lets you start it in text mode AND lets you qemu-system-i386 -drive file=debian-hurd-20150320.img,cache=writeback,format=raw --curses

interesting other options qemu-system-i386 -drive file=debian-hurd-20150320.img,cache=writeback,format=raw --curses -cpu core2duo -smp 2 -m 1G -enable-kvm --no-reboot

it's a core2duo processor treat this thing like 2 cpus memory 1 gigabyte enable kvm, which will make it SOOO much faster don't reboot the system. This avoids filesystem corruption

start in full screen -full-screen

qemu-system-i386 -drive file=debian-hurd-20150320.img,cache=writeback,format=raw --curses -cpu core2duo -smp 2 -m 1G -enable-kvm -full-screen

--kvm info on arch wiki https://wiki.archlinux.org/index.php/QEMU#Enabling_KVM https://wiki.archlinux.org/index.php/KVM

Checking support for KVM

Hardware support KVM requires that the virtual machine host's processor has virtualization support (named VT-x for Intel processors and AMD-V for AMD processors). You can check whether your processor supports hardware virtualization with the following command: $ lscpu Your processor supports virtualization only if there is a line telling you so. You can also run: $ egrep --color=auto 'vmx|svm|0xc0f' /proc/cpuinfo If nothing is displayed after running that command, then your processor does not support hardware virtualization, and you will not be able to use KVM. Note: You may need to enable virtualization support in your BIOS. Kernel support Arch Linux kernels provide the appropriate kernel modules to support KVM and VIRTIO. KVM modules You can check if necessary modules (kvm and one of kvm_amd, kvm_intel) are available in your kernel with the following command (assuming your kernel is built with CONFIG_IKCONFIG_PROC): $ zgrep CONFIG_KVM /proc/config.gz If the module is not set equal to y or m, then the module is not available. Para-virtualized devices

Para-virtualization provides a fast and efficient means of communication for guests to use devices on the host machine. KVM provides para-virtualized devices to virtual machines using the Virtio API as a layer between the hypervisor and guest. All virtio devices have two parts: the host device and the guest driver. VIRTIO modules Use the following command to check if needed modules are available: $ zgrep CONFIG_VIRTIO /proc/config.gz Loading kernel modules First, check if the kernel modules are automatically loaded. This should be the case with recent versions of udev. $ lsmod | grep kvm $ lsmod | grep virtio In case the above commands return nothing, you need to load kernel modules. Tip: If modprobing kvm_intel or kvm_amd fails but modprobing kvm succeeds, (and lscpu claims that hardware acceleration is supported), check your BIOS settings. Some vendors (especially laptop vendors) disable these processor extensions by default. To determine whether there's no hardware support or there is but the extensions are disabled in BIOS, the output from dmesg after having failed to modprobe will tell. List of para-virtualized devices network device (virtio-net) block device (virtio-blk) controller device (virtio-scsi) serial device (virtio-serial) balloon device (virtio-balloon) How to use KVM

Enabling KVM KVM must be supported by your processor and kernel, and necessary kernel modules must be loaded. See KVM for more information. To start QEMU in KVM mode, append -enable-kvm to the additional start options. To check if KVM is enabled for a running VM, enter the QEMU Monitor using Ctrl+Alt+Shift+2, and type info kvm. Note: If you start your VM with a GUI tool and experience very bad performance, you should check for proper KVM support, as QEMU may be falling back to software emulation. KVM needs to be enabled in order to start Windows 7 and Windows 8 properly without a blue screen.

I have to start qemu without curses in order to test if kvm has been enabled. I also probably need to check my bios setting. Perhaps my bios settings disable proper kvm support

I can add a new user with `adduser` joshua babbages

I can use msr-tools to write to my cpu to enable intel virtual technology rdmsr – read MSR from any CPU or all CPUs wrmsr – write values to MSR on any CPU or all CPUs THIS IS VERY DANGEROUS. directly writing to a cPU register could brick my system

cpuid – show identification and feature information of any CPU https://askubuntu.com/questions/103965/how-to-determine-if-cpu-vt-extensions-enabled-in-bios/103966#103966 http://linux.koolsolutions.com/2009/09/19/howto-using-cpu-msr-tools-rdmsrwrmsr-in-debian-linux/

mach comes with its own kernel debugger https://www.gnu.org/software/hurd/gnumach-doc/Kernel-Debugger.html https://www.gnu.org/software/hurd/microkernel/mach/gnumach/debugging.html

trouble shooting

Fixing corrupted filesystems

If you get logged into a root shell and your root filesystem is corrupted to fix it, is quite easy:

Use cat /etc/fstab to find the root filesystem, then run fsck.etx2 RootFileSystem~ Then run ~exit to shutdown.

If you are not able to access the root shell, then just run the command fix-hurd, that I've created .

Running fsck when you can't boot the machine

Also, it tells you that you will cause severe filesystem damage if you try to run the next command sh-2.05# fsysopts / -w sh-2.05# e2fsck /dev/hd0s1 press enter multiple times use loseupt -P -P forces the kernel to scan the partition table on a newly create loop device

The offset for partition one is 512 * 2048 = 1048576

losetup -o 1048576 --find --show hd0.img

The above command will print out the loop device that the partition in installed on. You can then run

fsck.ext2 /dev/loop0

These links seem to be some good information:

http://serverfault.com/questions/380186/how-to-run-fsck-on-guest-vms-from-kvm

http://www.ingent.net/en/tag/kvm/

here is an email explaining this as well.

cannot update errors

Insert CD errors

cannot update No public key KEY

add -cdrom SOMETHINGDEBIAN.iso to the qemu command

If apt-get update fails, then try the following command:


apt-get install debian-keyring
gpg --keyserver pgp.mit.edu --recv-keys 1F41B907
gpg --armor --export 1F41B907 | apt-key add -

Hurd refuses to boot properly

KEY is a BIG long string. In this case I've made it to be 1F41B907, but the string can be longer than that.


  ide0: reset: success
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }
  hd0: irq timeout: status=0x58 { DriveReady SeekComplete DataRequest }
  ide0: reset: success
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }
  end_request: I/O error, dev 03:01, sector 1100704
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }
  hd0: irq timeout: status=0x58 { DriveReady SeekComplete DataRequest }
  ide0: reset: success
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }
  hd0: irq timeout: status=0x58 { DriveReady SeekComplete DataRequest }
  ide0: reset: success
  hd0: status error: status=0x58 { DriveReady SeekComplete DataRequest }
  end_request: I/O error, dev 03:01, sector 5255176
  hd0: drive not ready for command
  hd0: irq timeout: status=0x50 { DriveReady SeekComplete }

Try disabling Hurd console:

/etc/default/hurd-console

You can still start it manually

How to start the Hurd console from the mach console (info here):

internet connectivity

You can reset the internet here: https://www.gnu.org/software/hurd/hurd/translator/pfinet.html

# console -d vga -d pc_mouse --repeat=mouse -d pc_kbd --repeat=kbd -d generic_speaker -c /dev/vcs

To find your gateway


ip r

default via 172.16.112.1 dev enp0s9 proto static metric 100
172.16.112.0/22 dev enp0s9 proto kernel scope link src 172.16.113.155 metric 100

172.16.112.1 is the gateway.

some qemu info

Network options

-nic [tap|bridge|user|l2tpv3|vde|netmap|vhost-user|socket][,...][,mac=macaddr][,model=mn] This option is a shortcut for configuring both the on-board (default) guest NIC hardware and the host network backend in one go. The host backend options are the same as with the corresponding -netdev options below. The guest NIC model can be set with model=modelname. Use model=help to list the available device types. The hardware MAC address can be set with mac=macaddr.

The following two example do exactly the same, to show how -nic can be used to shorten the command line length (note that the e1000 is the default on i386, so the model=e1000 parameter could even be omitted here, too):

qemu-system-i386 -netdev user,id=n1,ipv6=off -device e1000,netdev=n1,mac=52:54:98:76:54:32 qemu-system-i386 -nic user,ipv6=off,model=e1000,mac=52:54:98:76:54:32

-nic none Indicate that no network devices should be configured. It is used to override the default configuration (default NIC with "user" host network backend) which is activated if no other networking options are provided.

-netdev user,id=id[,option][,option][,...] Configure user mode host network backend which requires no administrator privilege to run. Valid options are:

id=id Assign symbolic name for use in monitor commands.

ipv4=on|off and ipv6=on|off

braunr apparently made vms use hugetlbfs for physical memory management?

This is good?

qemu stuff for vms

Trying out the vm image [2022-03-30 Wed]

https://www.gnu.org/software/hurd/public_hurd_boxen/sceen.html http://lifeisabug.com/kvm-virtualization-arch-linux-host-system-qemu-virtio-hugepages-systemd/

Add yourself to the kvm group. This group lets you start a kernel-ized virtual machine. You can check to which groups you belong with:


groups joshua

 http video audio


$ wget http://people.debian.org/~sthibault/hurd-i386/debian-hurd.img.tar.gz
$ tar -xz < debian-hurd.img.tar.gz

Then I went to read the readme: https://cdimage.debian.org/cdimage/ports/latest/hurd-i386/README


Make sure that you can access /dev/kvm to get full KVM speed (otherwise KVM will
be terribly slow). You can easily check with

      $ file -s /dev/kvm

that you properly get "ERROR: cannot read `/dev/kvm' (Invalid argument)",
which means that "file" was properly able to open kvm, just not smart enough
to use it :)

That's what happened to me. Let's run this monster!


$ qemu-system-i386 -m 4G -display curses -drive cache=writeback,file=./debian-hurd-20220226.img

So you're not gonna believe this, but it booted, then immediately rebooted. No idea why it needed to reboot. But this is the error message that I recieve after fsck fails:


/dev/hd0s2: Entry 'dmesg' in /var/log (8107) has deleted/unused inode 8808.  CLEARED.
/dev/hd0s2: Entry 'dmesg.1.gz' in /var/log (8107) has deleted/unused inode 8807.
  CLEARED.
/dev/hd0s2: Entry 'resolv.conf' in /etc (16193) has deleted/unused inode 17182.
 CLEARED.
/dev/hd0s2: Entry '.clean' in /tmp (72881) has deleted/unused inode 73676.  CLEA
RED.
/dev/hd0s2: Missing '.' in directory inode 98103.


/dev/hd0s2: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
        (i.e., without -a or -p options)
fsck exited with status code 4
failed (code 4).
An automatic file system check (fsck) of the root filesystem failed. A manual fs
ck must be performed, then the system restarted. The fsck should be performed in
 maintenance mode with the root filesystem mounted in read-only mode. ... failed
!
The root filesystem is currently mounted in read-only mode. A maintenance shell
will now be started. After performing system maintenance, press CONTROL-D to ter
minate the maintenance shell and restart the system. ... (warning).
Press Enter for maintenance

That's probably my fault! I was using termite, which is an upsupported terminal (it's developers tell you to use something else) and I was running the fish shell to start the hurd. Perhaps that just did something funky. And I don't feel like re-learn how to run fsck on the root filesystem. Also if I have to run fsck on the root filesystem at first boot, then it is probably best just to start over. This time I as using xfce4-terminal and bash only.


$ rm debian-hurd-20220226.img
$ tar -xz < debian-hurd.img.tar.gz
$ qemu-system-i386 -m 4G -display curses -drive cache=writeback,file=./debian-hurd-20220226.img

While it's booting let's go read up on some of the documentation:

Hey I found the actual hurd wiki that is more up to date! https://darnassus.sceen.net/~hurd-web/hurd/running/qemu/

Ahh! That page has some more tips on how to hurd the Hurd in qemu: Namely '-no-reboot'

Also when I started this time, I am pretty sure that it booted twice again. It did NOT leave me in a "you need to run fsck", but it did have a warning about the root filesystem not having enough room mounting /tmp. Also It did get to a login screen after what I think was a reboot. Again not entirely certain because I was half watching the boot up and looking at documentation.

BUT when it did get to the login screen


  Timeout reached while wating for return value
  /bin/console: Could not receive return value from daemon process: Connection tim
  ed out
  Starting enhanced syslogd: rsyslogdrsyslogd: could not load module 'imklog', err
  ors: trying to load module /usr/lib/i386-gnu/rsyslog/imklog.so: /usr/lib/i386-gn
  u/rsyslog/imklog.so: undefined symbol: klogWillRunPrePrivDrop [v8.39.0 try http:
  //www.rsyslog.com/e/2066 ]
  .
  Starting periodic command scheduler: cron.
  Can't start system message bus - /proc is not mounted ... failed!
  Starting OpenBSD Secure Shell server: sshd.

  Debian GNU/Hurd bookworm/sid debian console

  login:

I cannot login. I am typing root, and RET, but nothing happens. The screen appears to be froze. Ok. I am going to assume that it rebooted, which I have heard is a great way for the Hurd to get filesystem corruption. So let's try again. My host machine is Guix System running sway.


$ rm debian-hurd-20220226.img
$ tar -xz < debian-hurd.img.tar.gz
$ qemu-system-i386 -m 4G -display curses -drive cache=writeback,file=./debian-hurd-20220226.img -no-reboot

Ok! This time I watched the boot process and did not look away. 1st I got a warning that the root filesystem did NOT have enough space and /tmpfs was going to be mounted. Later I got a warning that the system needed to reboot, because ext2fs had died. Then it tried to reboot, and qemu would not let it. So I guess it halted.

This is what I see after it has halted:


joshua@crazyhorse ~/prog/gnu/hurd/vm$ qemu-system-i386 -m 4G -display curses -drive cache=writeback,file=./debian-hurd-20220226.img -no-reboot
WARNING: Image format was not specified for './debian-hurd-20220226.img' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.

Well I suppose that there should NOT be any filesystem corruption...since it did not reboot. Let's try specifiying format raw.


$ qemu-system-i386 -m 4G -display curses -drive format=raw,cache=writeback,file=./debian-hurd-20220226.img -no-reboot

Ok root filesystem seems to have some curruption. Here is what I see:


INIT: No inittab.d directory found
Using makefile-style concurrent boot in runlevel S.
mount: /proc already mounted
chmod: changing permissions of '/dev/shm': Read-only file system
Activating swap...done.
Checking root file system.../dev/hd0s2 was not cleanly unmounted, check forced.
/dev/hd0s2: Deleted inode 17353 has zero dtime.  FIXED.
/dev/hd0s2: Entry 'xconsole' in /dev (8109) has deleted/unused inode 8809.  CLEA
RED.
/dev/hd0s2: Unattached inode 9422


/dev/hd0s2: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
        (i.e., without -a or -p options)
fsck exited with status code 4
failed (code 4).
An automatic file system check (fsck) of the root filesystem failed. A manual fs
ck must be performed, then the system restarted. The fsck should be performed in
 maintenance mode with the root filesystem mounted in read-only mode. ... failed
!
The root filesystem is currently mounted in read-only mode. A maintenance shell
will now be started. After performing system maintenance, press CONTROL-D to ter
minate the maintenance shell and restart the system. ... (warning).
Press Enter for maintenance
(or press Control-D to continue):

Again, if I am having to fsck a root filesystem before adding in a regular user, then it is probably time to start over again. I am trying to brainstorm what could be the problem...Is the 5G image too small? Does the Hurd need to start distribting larger qemu images? Would a debian/linux qemu image of 5G be enough?

This time I will try another terminal emulator: qterminal


$ rm debian-hurd-20220226.img
$ tar -xz < debian-hurd.img.tar.gz
$ qemu-system-i386 -m 4G -display curses -drive format=raw,cache=writeback,file=./debian-hurd-20220226.img -no-reboot

Ok, I had the same issue. A warning message appeared that said something like "root filesystem has insufficient free space mounting tmpfs or tmp". Then later on it said something about the ext2fs translator had died, so it needed to reboot. Then it tried to reboot, and qemu would not let it. The last time I tried to reboot an image that had to reboot because the ext2fs translator had died it had filesystem corrucption and told me to run fsck. Let's just restart again.

Is kvm kernel module loaded?


$ modprobe kvm_intel


$ rm debian-hurd-20220226.img
$ tar -xz < debian-hurd.img.tar.gz
$ qemu-system-i386 -m 4G -display curses -drive format=raw,cache=writeback,file=./debian-hurd-20220226.img -no-reboot

So I have been seening this 800 by 600 graphic mode in blue letters at start up. I get the same warning about the root filesystem not having enough space, ext2fs crashes, and it tried and fails to reboot. Is -display curses causing stuff to not work? I suppose that is possible. BUT I do not know how I can use the Hurd on my laptop, which has a physical dvorak layout... Let's try without curses...


$ rm debian-hurd-20220226.img
$ tar -xz < debian-hurd.img.tar.gz
$ qemu-system-i386 -m 4G -drive format=raw,cache=writeback,file=./debian-hurd-20220226.img -no-reboot

Nope. -display curses is NOT the problem. I tried booting a freshly created image again, and I went through the same whoopsie daisy again: root filesystem does not have enough space, ext2fs crashes, and it tries to reboot. I suppose since I am on Guix system, I could try using the hurd vm image that Guix supports. But I know it is a little dated.

Anyway, I have a little side project that I am currently working on for Guix System (an opensmtpd configuration). I think I will spend the rest of the day working on that instead of trying to get a Hurd vm working. I am planning on setting up an Dell Optiplex 7020 server in a few days. Maybe that is old-ish enough to run the Hurd in real hardware. We'll find out!

Trying out the Hurd part 2 [2022-03-30 Wed]

I posted the above LONG email to bug-hurd, and I got this email from someone:


I faced that problem with the latest image. What I did was to resize the image with +5GB see
https://cdimage.debian.org/cdimage/ports/latest/hurd-i386/README.txt

The issue seems that the latest image missed space for swap out or for another reason.
I don't know if diagnosis is correct or I was lucky.

Try to increase the image size to see it solves your issue.

Well let's give that a try!


qemu-img resize debian-hurd*.img +50G

Image resized.


$ parted debian-hurd*.img
(parted) resizepart 2 100%
(parted) quit


sudo losetup -o $((512*1953792)) /dev/loop0 debian-hurd*.img
sudo resize2fs /dev/loop

resize2fs 1.46.4 (18-Aug-2021)
Resizing the filesystem on /dev/loop0 to 14143232 (4k) blocks.
The filesystem on /dev/loop0 is now 14143232 (4k) blocks long.


sudo losetup -d /dev/loop0

Well let's try running it again


ls -lha  # the img is now 55G

total 1.9G drwxr-xr-x 2 joshua users 4.0K Mar 30 13:33 . drwxr-xr-x 3 joshua users 4.0K Mar 30 09:16 .. -rw-r--r-- 1 joshua users 55G Mar 30 14:15 debian-hurd-20220226.img -rw-r--r-- 1 joshua users 386M Feb 26 05:50 debian-hurd.img.tar.gz


$ qemu-system-i386 -m 4G -display curses -drive format=raw,cache=writeback,file=./debian-hurd-20220226.img -no-reboot

This time boot worked! For a second I was actually able to type too! Then I switch to a different virtual desktop to copy the output of boot. When I switched back to try to press return, the Hurd seemed to freeze. I see =login: root=, but trying to type or press enter does not appear to change anything.


Timeout reached while wating for return value
/bin/console: Could not receive return value from daemon process: Connection tim
ed out
Starting enhanced syslogd: rsyslogdrsyslogd: could not load module 'imklog', err
ors: trying to load module /usr/lib/i386-gnu/rsyslog/imklog.so: /usr/lib/i386-gn
u/rsyslog/imklog.so: undefined symbol: klogWillRunPrePrivDrop [v8.39.0 try http:
//www.rsyslog.com/e/2066 ]
.
Starting periodic command scheduler: cron.
Can't start system message bus - /proc is not mounted ... failed!
Starting OpenBSD Secure Shell server: sshd.

It probably NOT a good sign that proc is not mounted. I think I may have actually made a mistake when I was resizing the image. And perhaps since I am running on sway, and the Hurd is running on a terminal that only runs in X is causing some issues. Anyway I'll try again with terminal sakura.


 rm debian-hurd-20220226.img
 tar -xz < debian-hurd.img.tar.gz
 qemu-img resize debian-hurd*.img +50G

Image resized.


parted debian-hurd*.img
(parted) resizepart 2 100%
(parted) quit


sudo losetup -o $((512*1953792)) /dev/loop1 debian-hurd*.img
sudo resize2fs /dev/loop1

resize2fs 1.46.4 (18-Aug-2021)
Resizing the filesystem on /dev/loop0 to 14143232 (4k) blocks.
The filesystem on /dev/loop0 is now 14143232 (4k) blocks long.


sudo losetup -d /dev/loop1


ls -lha

total 1.9G drwxr-xr-x 2 joshua users 4.0K Mar 30 14:29 . drwxr-xr-x 3 joshua users 4.0K Mar 30 09:16 .. -rw-r--r-- 1 joshua users 55G Mar 30 14:49 debian-hurd-20220226.img -rw-r--r-- 1 joshua users 386M Feb 26 05:50 debian-hurd.img.tar.gz

Samuel mentioned that we really should have this --enable-kvm option


 qemu-system-i386 -m 4G -display curses -drive format=raw,cache=writeback,file=./debian-hurd-20220226.img -no-reboot --enable-kvm

Ok that seems to work!

Let's change the root password in the vm


passwd

That worked. Let's reboot and try to get it so that I can ssh into the vm.


halt

Ok halt seems to have broken the machine...It is now just sitting there. C-c and C-d is doing nothing. I cannot get back to a prompt. And the hurd does not seem to be shutting down. What is the command to turn poweroff the hurd? =poweroff=? shutdown?

Ahh the README has this info


One can safely shut down the system with

      $ sudo poweroff

Ok, the hurd does NOT seem to be halting...So I am going to force close the terminal window and see what happens. I will also try using the same image, but there's a good chance that I am going to get filesystem corruption. And #guix irc mentioned that I should try out foot as a minimal wayland only terminal.


qemu-system-i386 -m 4G -display curses -drive format=raw,cache=writeback,file=./debian-hurd-20220226.img -no-reboot --enable-kvm

Well apparently halt + me force closing the terminal window caused some filesystem corruption. At bootup, the Hurd said that it noticed some issues with the root filesystem that it tried to correct. It was unable to mount it. It tried to do a reboot, which of course failed because qemu won't let it (because rebooting the Hurd causes filesystem corruption).

Well let's try this again...

:DeletingHurdImageAndReInstalling:


 rm debian-hurd-20220226.img
 tar -xz < debian-hurd.img.tar.gz
 qemu-img resize debian-hurd*.img +50G

Image resized.


parted debian-hurd*.img
(parted) resizepart 2 100%
(parted) quit


sudo losetup -o $((512*1953792)) /dev/loop0 debian-hurd*.img
sudo resize2fs /dev/loop0

resize2fs 1.46.4 (18-Aug-2021)
Resizing the filesystem on /dev/loop0 to 14143232 (4k) blocks.
The filesystem on /dev/loop0 is now 14143232 (4k) blocks long.


sudo losetup -d /dev/loop0


ls -lha

total 1.9G drwxr-xr-x 2 joshua users 4.0K Mar 30 15:10 . drwxr-xr-x 3 joshua users 4.0K Mar 30 09:16 .. -rw-r--r-- 1 joshua users 55G Mar 30 15:11 debian-hurd-20220226.img -rw-r--r-- 1 joshua users 386M Feb 26 05:50 debian-hurd.img.tar.gz

:END:

maybe this time I will try to use the correct options for networking.


 qemu-system-i386 -m 4G -net user,hostfwd=tcp:127.0.0.1:2222-:22 -net nic,model=e1000 -display curses -drive format=raw,cache=writeback,file=./debian-hurd-20220226.img -no-reboot --enable-kvm

Login worked. Changing the root password worked. ping -c 3 gnu.org failed. 100% packets lost. I did run ping as root, which should work right?

adduser joshua -> -bash: adduser: command not found

As the demo user sudo poweroff works.

Let's power it back on and see if I can ssh into the box. Am I doing it wrong?


joshua@crazyhorse ~ (master)> ssh localhost:2222
ssh: Could not resolve hostname localhost:2222: Name or service not known
joshua@crazyhorse ~ (master) [255]> ssh 127.0.0.1:2222
ssh: Could not resolve hostname 127.0.0.1:2222: Name or service not known
joshua@crazyhorse ~ (master) [255]> ssh 127.0.0.1
ssh: connect to host 127.0.0.1 port 22: Connection refused
joshua@crazyhorse ~ (master) [255]> ssh 127.0.0.1
ssh: connect to host 127.0.0.1 port 22: Connection refused

ping -c 3 gnu.org still is not working. There is some tips about how to get networking to work here: https://www.debian.org/ports/hurd/hurd-install, https://darnassus.sceen.net/~hurd-web/hurd/running/qemu/#index7h1, and https://darnassus.sceen.net/~hurd-web/hurd/running/qemu/networking/ Where should I start?

It looks like dhcp should work? Is that running?

It looks like pfinet does the TCP stuff. Is that running?

/servers/socket/2 does exist...

The guides online suggest to make sure /etc/resolv.conf is correct.


cat /etc/resolv.conf

Looks good!

this guide says that eth0 is in /dev/eth0 and eth0 does exist! as well as eth1, eth2, eth3.


The Debian way is supported starting from sysvinit 2.88dsf-48 and hurd
1:0.5.git20140320-1: /etc/network/interfaces is used like on Linux. The only
difference is that network boards appear in /dev, and interfaces should thus be
specified as /dev/eth0 etc.

It looks like /etc/network/interfaces is an important file in the hurd


cat /etc/network/interfaces

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto /dev/eth0
iface /dev/eth0 inet dhcp

Trying out the Hurd part 3 [2022-04-01 Fri]

Well I should take a look at man 5 interfaces apparently.


qemu-system-i386 -m 4G -display curses -drive format=raw,cache=writeback,file=./debian-hurd-20220226.img -no-reboot --enable-kvm

I logged in as the demo user.


demo@debian $ ping -c 3 gnu.org

PING gnu.org (209.51.188.116): 56 data bytes
--- gnu.org ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss

Does the demo user on the Hurd image have netorking set up? It looks like it is at least trying.

My host OS has networking, albeit it is a bit slow.

I poweroff ed the Hurd, and restarted it.


qemu-system-i386 -m 4G -display curses -drive format=raw,cache=writeback,file=./debian-hurd-20220226.img -no-reboot --enable-kvm


demo@debian $ ping -c 3 gnu.org

PING: unknown host

Well that's not an improvement...I know that Samuel just updated the Hurd image to fix the latest ext2fs crash right after boot. I suppose that I should try that.

ls

debian-hurd-20220226.img
debian-hurd.img.tar.gz


rm *
wget http://people.debian.org/~sthibault/hurd-i386/debian-hurd.img.tar.gz
tar -xz < debian-hurd.img.tar.gz

--2022-04-01 10:51:03--  http://people.debian.org/~sthibault/hurd-i386/debian-hurd.img.tar.gz
Resolving people.debian.org (people.debian.org)... failed: Name or service not known.
wget: unable to resolve host address ‘people.debian.org’

Well, now my host OS is having some networking issues, which is weird because I am using the ethernet port on my ThinkPad T400.


ping -c 3 gnu.org

PING: unknown host


sudo herd restart networking

And I am back to the same problem on my host OS:


ping -c 3 gnu.org

PING: unknown host

Ok, networking issues on my host OS are gone. Let's try again.


rm *
wget http://people.debian.org/~sthibault/hurd-i386/debian-hurd.img.tar.gz
tar -xz < debian-hurd.img.tar.gz

ls

debian-hurd-20220331.img
debian-hurd.img.tar.gz

The date on the image has changed. That's a good sign!


qemu-system-i386 -m 4G -display curses -drive format=raw,cache=writeback,file=./debian-hurd-20220331.img -no-reboot --enable-kvm

In the hurd I am logging in as 'demo' user. I think I saw an warning message that /proc was not mounted.

ping -c 3 gnu.org

PING gnu.org (209.51.188.116): 56 data bytes= =--- gnu.org ping statistics ---= =3 packets transmitted, 0 packets received, 100% packet loss


 qemu-img resize debian-hurd*.img +50G

Image resized.


parted debian-hurd*.img
(parted) resizepart 2 100%
(parted) quit


sudo losetup -o $((512*1953792)) /dev/loop1 debian-hurd*.img
sudo resize2fs /dev/loop1

resize2fs 1.46.4 (18-Aug-2021)
Resizing the filesystem on /dev/loop0 to 14143232 (4k) blocks.
The filesystem on /dev/loop0 is now 14143232 (4k) blocks long.


sudo losetup -d /dev/loop1


ls -lha

total 3.2G drwxr-xr-x 2 joshua users 4.0K Apr 1 16:09 . drwxr-xr-x 3 joshua users 4.0K Mar 30 09:16 .. -rw-r--r-- 1 joshua users 55G Apr 2 17:28 debian-hurd-20220331.img -rw-r--r-- 1 joshua users 410M Mar 31 01:36 debian-hurd.img.tar.gz

Trying out the Hurd part 4 [2022-04-08 Fri]

So, I found out from Samuel's email, that ping -c 3 gnu.org is not the most reliable method to determine if the Hurd vm has network connectivity. Sometimes the host OS will only let root processes emit icmp packets.


sudo apt-get update

This actually seems to connect to the network and resync stuff.


sudo apt-get ugrade

It looks like I am updating debian sid...It seems to work until, an error message that says:

E: failed to

Well let's try updating another way.


sudo apt --list-upgradable

There's a samba package apparently, which I do not intend to use.


sudo apt remove samba-libs

Setting up a wm in the Hurd

Goals: dwm is probably the best bet as it is the absolute simplest software out there.

Goals	wm
	ratpoison	i3	dwm
Simple (only written in C)	yes	yes	yes
Maintained	yes	yes	yes
Easy to Use	limited mouse support	yes	maybe
written for X	yes	yes	yes

youpi, who is the main hurd maintainer just said on irc that X support in the Hurd is currently broken. [2022-07-05 Tue]

booting the Hurd from grub. IE: grub normally boots Linux

So there is not much chance of getting X to work until someone fixes it.


grub> multiboot /boot/gnumach.gz root=device:hd0s6
grub> module  /hurd/ext2fs.static ext2fs --readonly \
                   --multiboot-command-line='${kernel-command-line}' \
                   --host-priv-port='${host-port}' \
                   --device-master-port='${device-port}' \
                   --exec-server-task='${exec-task}' -T typed '${root}' \
                   '$(task-create)' '$(task-resume)'
grub> module /lib/ld.so.1 exec /hurd/exec '$(exec-task=task-create)'
grub> boot

building Hurd gnumach and MIG

hurd/compiling Hurd

I seem unable to boot the Hurd. My computer's bios doesn't like it. If my bios doesn't like it, I don't think that grub can boot from it . https://www.gnu.org/software/hurd/hurd/building.html

gnumach

building the Hurd: apt-get build-dep hurd hurd-2017-02-28:21:25:56< teythoon> and the same for gnumach if you want to build that hurd-2017-02-28-21:26:01< teythoon> then clone the repository hurd-2017-02-28-21:26:08< teythoon> autoreconf -fi hurd-2017-02-28-21:26:22< teythoon> the usual ./configure, make dance hurd-2017-02-28-21:26:46< teythoon> building it shouldn't be too problematic hurd-2017-02-28-21:27:02< teythoon> building something that boots is a bit more tricky hurd-2017-02-28-21:27:08< teythoon> good luck ;) https://www.gnu.org/software/hurd/microkernel/mach/gnumach/building.html

mig

developing notes

hurd dev notes

Make clean will remove previous builds and possible rebuild everything... I'll see the errors again.

pflocal is for /server/socket/1 pfinet runs on /server/socket/2 and is for networking ipv6 runs on /server/socker/26

hostpriv means gnumach usually a function that starts with vm_ will normally be found with mach, because mach manages memory the proc server knows about processes and their UIDs. This is needed because gnumach does not understand UIDs. It doesn't understand which process is root or a user UID 0 is root PROC is a macro that says this is proc EPERM macro means not authorized or not root. proc is started at bootstrap. It allows processes root if they have UID 0 this lets a user to create a subhurd. libshouldbeinlibc is the library where unstable functions that should be in libc should be kept.

assign copyright to the FSF

Please email the following information to assign@gnu.org, and we will send you the assignment form for your past and future changes.

Please use your full legal name (in ASCII characters) as the subject line of the message. ---------------------------------------------------------------------- REQUEST: SEND FORM FOR PAST AND FUTURE CHANGES

[What is the name of the program or package you're contributing to?]

[Did you copy any files or text written by someone else in these changes? Even if that material is free software, we need to know about it.]

[Do you have an employer who might have a basis to claim to own your changes? Do you attend a school which might make such a claim?]

[For the copyright registration, what country are you a citizen of?]

[What year were you born?]

[Please write your email address here.]

[Please write your postal address here.]

[Which files have you changed so far, and which new files have you written so far?]

mach

mach IPC

good for RPC, OO distributed programming, streaming of data, and sending lots of data.

boot with a modified mach

just apt build-dep gnumach hurd-2017-06-26:14:36:23< teythoon> then clone gnumach hurd-2017-06-26-14:37:20< teythoon> then configure it like this: ../configure --disable-wireless-group --disable-scsi-group --disable-floppy --disable-pcmcia-group --enable-kdb --enable-net-group hurd-2017-06-26:14:37:36< teythoon> make, copy gnumach to /boot, update-grub, and reboot

I can compile the texinfo source with texi2pdf .texi

how to create a git diff

how to write a change log

roadmap

hurd roadmap

fix libdiskfs locking issues

git diff > patch.diff Note that there can be many commits between the two hashes. ie: one commit is from 2 months ago, and the second commit is from today https://www.gnu.org/software/hurd/source_repositories.html https://www.gnu.org/prep/standards/html_node/Change-Logs.html

These days, the ext2fs server seems to cause most of the crashes on the Hurd. It is believed that libdiskfs causes most of these issues. A unit testing framework that checks locking in various code paths or a system that constantly checks locking correcting at runtime could help solve this issue.

This is a fairly involved project that must optimize locking issues in a multithreaded application.

improve I/O performance

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/libdiskfs_locking.html

One obvious reason that the Hurd feels slow when compared to GNU/Linux is a slow I/O system performance, specifically very slow hard disk access. The first task is to implement read-ahead, which tries to read more information from disk than in needed presently. Then the excess data can be retrieved from RAM. Even a simple read-ahead would be a huge performance improvement.

Other related tasks include scheduling reads and writes to minimize head movement, effective block caching, reading and writing multiple blocks at once, effective reads/writes to partial blocks, and various ext2 filesystem server optimizations.

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/disk_io_performance.html

Read-ahead is an involved project, though not a research project.

porting valgrind to the hurd

Valgrind is an important debugging tool that checks for memory errors. It can help discover existing bugs in the hurd servers as well as aid in porting programs to the Hurd.

There is an unfinished start of port to the Hurd, and it is not an easy task. However, it is a fairly predictable one. One must learn how valgrind handles system calls and the Hurd RPCs. No major design or research is necessary.

new driver framework

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/valgrind.html The Hurd currently uses some hardware drivers in the mach kernel. These drivers are old, which potentially makes the Hurd have a hard time supporting newer machines. The most promising solution is to implement drivers from the rump kernel, which uses netbsd code.

This is a fairly involved project that will probably require previous device driver experience.

kernel instrumentation

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/driver_glue_code.html The Hurd suffers from poor performance, and while its developers know many of the problems that cause poor performance, they need better tools to diagnosis performance bottlenecks. The Hurd has some existing kernel probes, but not all of the patches were functional and only some were accepted into the project. The most reasonable solution is to improve the existing kernel probes and a nice frontend for writing scripts to control the probes.

This is an involved project that requires knowledge of low level programming. It requires a good knowledge of gnumach and interactions of complex systems.

file locking support

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/dtrace.html UNIX has several different file locking mechanisms. Some of them work on the hurd while others are buggy or non-existent. This breaks numerous applications. Ideally all file locking mechanisms should work properly. This is not a trivial task and implementing or correcting existing file locking will require a competent developer.

NEWBIE IDEAS

improve glibc for the hurd and fix compatibility problems

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/file_locking.html https://www.gnu.org/software/hurd/community/gsoc/project_ideas/testsuites.html

gnumach code clean up

The Hurd uses glibc to forward POSIX requests to various Hurd RPCs protocals. The Hurd is missing several POSIX and other standards. Some of these tasks can be handled by less experienced developers, and others will require immense work. This might be a good area to begin Hurd development. https://www.gnu.org/software/hurd/community/gsoc/project_ideas/gnumach_cleanup.html

The Hurd runs on top of the gnumach microkernel, which is in a fairly bad shape. Gnumach is a modified version of the research CMU Mach, which has bits of BSD code. Parts of CMU Mach ripped out many UNIX mechanisms to be replaced by userspace processes. The result is a code base that is hard to change, difficult to support modern hardware, or even to fix bugs.

automated testing

One can begin by fixing all of the compiler warnings, restructuring code, removing dead or unused code paths, etc. You should have a good knowledge of C and experience in working in large C code bases. This is not necessarily a task that requires a lot of research, but it is probably a lot of busy work and careful changes to the existing code. https://www.gnu.org/software/hurd/community/gsoc/project_ideas/testing_framework.html

https://www.gnu.org/software/hurd/open_issues/unit_testing.html

Large complex software often employs a testing framework to reliably fix bugs and ensure that no bug appears twice. Ideally, when a developer fixes a bug, he would write a test to ensure that the bug would not occur again. This also applies when adding a new feature. Since there is no testing framework, it's harder to ensure that the Hurd does not add a new bug with a new feature.

This project requires a knowledge of C build systems and unit frameworks, but one need not be a Hurd expert to complete this project. This is an excellent project to get one started on Hurd development.

improve the GDB port for the hurd

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/gdb.html

GDB is a helpful debugging tool for the Hurd, but the Hurd port has some bugs and lacks some features. You can help by fixing bugs and implementing some of the features like gdbserver. It would also be nice if GDB could properly debug a server that calls another server. For example if you are debugging the ext2fs server, gdb should be able to follow code execution that uses libdiskfs.

usb support/sound support

You will need to learn how GDB works generally, how the Hurd port works, and some general knowledge of the Hurd and gnumach.

Many new laptops today lack a CD drive, which makes it harder to install the Hurd, and USB support is an important feature. The recommended way to fix this is to run a rump kernel in a userspace process. The rump kernel would support the usb hardware, and the Hurd would do the rest.

The Hurd has initial sound support, but it could be improved. The initial patches work, but they are not the perfect solution. An ideal solution allows changing the initial code to be more Hurdish. You can improve the current implementation or fix any existing bugs.

Guix/Hurd

startup in scheme

This requires good knowledge of the Hurd and gnumach as well as experience programming in C.

Currently only Debian GNU/Hurd is the only distribution that runs the GNU Hurd. While the Hurd works well on debian, many developers want to see a GNU GuixSD/Hurd. GNU Guix is GNU's new package manager with many forward looking features. While GNU Guix already runs on the Hurd, the Hurd's startup is still based on a Debian approach. GNU GuixSD uses the init system Shepherd to boot up. Shepherd needs to be ported over to work on the Hurd.

gnu package manager guix

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/package_manager.html

Most package managers use non-trivial databases to help one easily install, delete, and update software. The Hurd can use a much better solution: stowfs. Stowfs can create a virtual directory tree, on the fly, without using any global databases. This can ease the development burden of maintaining guix on the Hurd.

64 bit

This project includes improving GNU Guix, GNU's package manager to the Hurd. It will require knowledge of gnu guile and C. One will have to modify GNU Guix to use stowfs.

More processors are 64 bit processors, and users expect the benefits of 64 bit processors. Unfortunately, the Hurd only runs 32 bit. This is not a huge issue, because 64 bit machines still can run in 32 bit mode. However, the Hurd should eventually be able to run in 64 bit mode.

make some stub libraries

Because of some deficiencies in gnumach, a 64 bit kernel space and a 64 bit user space is a little too difficult to implement. An easier fix is to run the Hurd in a 64 bit kernel space and a 32 bit userspace process. The kernel can already run in a 64 bit, but more work is needed for a 64 bit to 32 bit conversion. https://www.gnu.org/software/hurd/community/gsoc/project_ideas/hardware_libs.html

Numerous programs link to various libraries to provide ranging hardware support. Examples include libusb and libbluetooth. The Hurd doesn't support these devices, but the programs could still be built if the Hurd had dummy libraries returning appropriate error codes (ie: no device found). Many of these programs would still be useful without the ability to access usb or bluetooth.

cd audio grabbing

The goal is to get a dummy library built for the Hurd and package it in Debian GNU/Hurd. This requires learning a bit about the libraries and debian packaging, but it does not require intense research. https://www.gnu.org/software/hurd/community/gsoc/project_ideas/cdparanoia.html

The hurd supports CD-ROMs, but not for extracting audio files, which means that many applications will not build on the Hurd. Adding support for audio extraction should not be too hard. It will involve learning about the Hurd and ATAPI commands, but it is certainly doable.

improve perl/python support

https://www.gnu.org/software/hurd/community/gsoc/project_ideas/perl_python.html

The Hurd supports perl and python, but the implementation is not complete and both language's test suite report some failures. One can work on fixing these test suite failures. Perl is more important and has more failures. This will involve a fair amount of C programming and may mean diving deeply into the Hurd's internals.

DONE physical memory management buddy allocator

State "DONE" from [2017-04-23 Sun 14:35]

DONE implement xattr support

check out this page:

State "DONE" from [2017-04-23 Sun 14:35]

https://git.savannah.gnu.org/cgit/hurd/hurd.git/plain/TODO

gnumach roadmap

file:///home/joshua/programming/gnu/hurd/vm/web-page/home/joshua/programming/web.rendered/microkernel/mach/gnumach/projects.html

Improve Mach's IPC mechanism: Mach 5

The Mach IPC mechanism has some deficiencies, many of which cause mach's poor performance.

Type Descriptor rework

A mach message contains pairs for type descriptors and values. Each type descriptors describes the kind of data and the amount of data. The kernel needs to parse the message to rewrite rights and pointers. The current format describes a type followed by a value, then another type followed by a value, etc.

The change to the format involves moving the type descriptors to the beginning of the message, which will create an index into the data. One type of type descriptors is machmsgtypelongt, which is unneeded. This type descriptors can be removed.

Faster Syscall Interface

Luckily the Mach4 type descriptor contains one unused bit. This bit can be used to indicate that the message is of type Mach 5, which would provide an easy transition to the new ABI. Currently gnumach uses trap gates to enter in the kernel on i386. Another solution is to use sysenter, which is not available on all platforms. Linux fixes this by using the VDSO mechanism, which we can do too. We believe that implementing sysenter/sysexit, mach's IPC could have a 10% boost in performance.

There is an existing prototype.

Interface for userspace drivers

https://lists.gnu.org/archive/html/bug-hurd/2015-05/msg00000.html https://lists.gnu.org/archive/html/bug-hurd/2016-09/msg00056.html The rump kernel seems to be the best option for this. There is a rump kernel prototype that has been implemented. It just needs to be tweaked to be more hurdish.

https://lists.gnu.org/archive/html/bug-hurd/2016-02/msg00126.html

gnu MIG roadmap

gnumach is an IDL compiler. It takes a specification and generates code that simplifies the remote procedure calls. It has some issues.

I can get more ideas from here: file:///home/joshua/programming/gnu/hurd/vm/web-page/home/joshua/programming/web.rendered/tag/open_issue_mig.html

GNU MIG is not good at sending structured data

file:///home/joshua/programming/gnu/hurd/vm/web-page/home/joshua/programming/web.rendered/open_issues/mig_portable_rpc_declarations.html

Some servers want to be able to send lots of complex data. Imagine a server that sends email using a remote smtp server. To send the email, the hurd server needs your email, password, and connection information. All this information ought to be sent in some type of data structure. MIG is not suited for this, nor can it do this.

MIG ought to be modified to allow for handling arbitrarily complex data structures. This may not be an easy task, and it might require a lot of research to find the best solution.

Duplicate inclusion guards

file:///home/joshua/programming/gnu/hurd/vm/web-page/home/joshua/programming/web.rendered/open_issues/duplicate_inclusion_guards.html

MIG does not call functions on when the reply is an error

MIGs variable length strings support is broken

What is low level serialization?

file:///home/joshua/programming/gnu/hurd/vm/web-page/home/joshua/programming/web.rendered/open_issues/mig_error_reply.html

MIG apparently tries to do serialization/parsing and message sending/receiving, which I guess should be separate? file:///home/joshua/programming/gnu/hurd/vm/web-page/home/joshua/programming/web.rendered/microkernel/mach/mig/documentation.html

some hurd tasks

Braunr plans to do that in x15

port packages to guix!

TODO some things I can do now

hurd

get rid of p-thread to do task

hurd.texi

?

learn argp in C, this will help me with using trivfs

learn some C

State "TODO" from [2021-01-26 Tue 09:56]

A struct is a collection of things in one variable name.

learn some gnu build system. This will help me with porting

update the project ideas page

A union is a collection of things in one variable name, BUT only one thing may have data in it at a time. https://www.gnu.org/software/hurd/community/gsoc/project_ideas.html

More ideas are here: https://www.mail-archive.com/debian-hurd@lists.debian.org/msg25179.html

maybe I can test this deadlock issue:

https://darnassus.sceen.net/~hurd-web/open_issues/ext2fs_deadlock/

take MIT open courseware courses

https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-s096-effective-programming-in-c-and-c-january-iap-2014/

https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-171-software-engineering-for-web-applications-fall-2003/

https://ocw.mit.edu/courses/comparative-media-studies-writing/cms-611j-creating-video-games-fall-2014/

https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-033-computer-system-engineering-spring-2009/

All the online text books!

when the Hurd gets a new feature I can mention it the qoth next page

https://ocw.mit.edu/courses/online-textbooks/#electrical-engineering-and-computer-science

improve the Hurd users guide

This is the page where the Hurd talks about new features that it got. https://www.gnu.org/software/hurd/users-guide/using_gnuhurd.html

help Almundena Garcia is his SMP work

I can delete the section dealing with CVS

PATCHSMP initialization: detection and enumeration OFF TOPIC PRAISE]]

sell old laptops that can run the Hurd natively

https://github.com/AlmuHS/GNUMach_SMP/tree/smp-new :PROPERTIES: :ID: 1e47bf17-13e2-4cde-9bd5-c92fd0fb6f45 :END:

I burnt the cd to physical media and tried to install natively on X220 thinkpad.

We have an issue because the SSD is not detected by gnumach, but still hogs IRQ 10, so when rumpdisk tries to enumerate disks from the ramdisk image, it cannot gain control of the disks.

Can we add to gnumach an optional command line flag "disk=off" so the kernel does not probe any disk controllers, I think that would enable the installer to proceed on my hardware.

Damien

On 26/1/21 4:00 pm, Damien Zammit wrote: > I burnt the cd to physical media and tried to install natively on X220 thinkpad.

I also tried the same image on a physical X230 thinkpad, this time the disk was detected by gnumach, but it failed to detect the CD ROM so the installer could not proceed, and there was no way to make rumpdisk find the ATAPI drive because it was already controlled by gnumach.

I think the same solution of providing a manual override to boot with disk=off would also fix the installer on this hardware.

Damien Zammit, le mar. 26 janv. 2021 16:00:28 +1100, a ecrit: > Can we add to gnumach an optional command line flag "disk=off" so the kernel does not probe any disk controllers,

There is already such option in the debian package, as a debian-specific patch: noide

Samuel

This option disables the built-in disk drivers completely. Passing " nodisk" to gnumach command line will allow the user to boot the installer with a ramdisk, and rumpdisk should initialise AHCI controller in userspace.

This is a workaround for the installer for some modern hardware that the gnumach built-in kernel AHCI driver cannot drive. It is also useful for testing rumpdisk without kernel drivers.

--- linux/dev/drivers/block/genhd.c | 4 ~~++- 1 file changed, 3 insertions(~~), 1 deletion(-)

blk_dev_init();
extern char *kernel_cmdline;
if (!strstr(kernel_cmdline, " nodisk"))
blk_dev_init();

try out lwip

The lwip translator should be good enough for me for internet connectivity. It would be nice to give it a try.

# settrans -fgap /servers/socket/2 /hurd/lwip ↩ -i /dev/eth0 -a 192.168.0.50 -g 192.168.0.1 -m 255.255.255.0

add the necessary features to make /hurd/hello robust

trivfs_S_io_get_openmodes trivfs_S_io_clear_some_openmodes trivfs_S_io_set_some_openmodes trivfs_S_io_set_all_openmodes trivfs_S_io_readable trivfs_S_io_select

wiki

wiki things that need changed

DONE fix this broken link the relevant email is here.

DONE fix the pfinet and lwip that mention -i eth0 -> -i /dev/eth0

State "DONE" from [2019-01-04 Fri 07:28]
State "DONE" from [2019-01-07 Mon 14:16]

DONE add some info to what a firmlink is on page.

State "DONE" from [2019-01-07 Mon 16:24]


grep -C 20 -r firmlink | less

DONE on the lwip page show how to add find your IP address, netmask, and gateway

State "DONE" from [2020-10-01 Thu 16:32]

DONE in the qemu section, explain how to add a non-root user AND put them in the suoders file!

State "DONE" from [2020-10-01 Thu 16:03]

update the gnumach memory management issues pages

https://www.gnu.org/software/hurd/microkernel/mach/gnumach/memory_management.html

https://www.gnu.org/software/hurd/open_issues/gnumach_memory_management.html

fix qemu starting string.

https://www.gnu.org/software/hurd/open_issues/gnumach_memory_management_2.html Richard Braun, le ven. 28 août 2020 18:33:03 +0200, a ecrit: > On Sat, Aug 15, 2020 at 06:51:27PM +0200, Jan Wielkiewicz wrote: > > Also navigation is too complicated and messy, searching doesn't work at > > all, because https://darnassus.sceen.net/cgi-bin/hurd-web is dead half > > of the time. > > Any special wishes? > > By the way, the main reason why that website is so often unreachable is > netdde freezing. I've restored an old trick I used in the past to mitigate > the problem, which is merely a cron script that kills it every hour. > > If you want that to improve, someone will have to hunt that bug down.

One way to avoid the bug is to use the e1000 hardware network type. I guess the rtl8139 driver that we ship in netdde has a bug in its irq handling (we had already seen that kind of bug a long time ago in the IDE driver). Moving to rump-based network drivers will probably help fixing this kind of issue long-term :)

add how to set up dhcp in qemu

http://darnassus.sceen.net/~hurd-web/hurd/running/debian/dhcp/

My /etc/network/interfaces/ file in hurd looks like this:

#+BEGIN_SRC sh :results output :exports both

# This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface auto lo iface lo inet loopback

# The primary network interface auto /dev/eth0 iface /dev/eth0 inet dhcp #+END_SRC

make sure the title on the website says GNU/Hurd not just GNU

add in some documentation about why the Hurd has ext2 and not ext4

Email from Benjamin Cardinal: {bug #56000} Website postfix i

add in example of how to use xmlfs

Email from Jeffrey Walton: Re: Debian GNU/Hurd 2019 relea The xmlfs documentation can be found here: http://cvs.savannah.gnu.org/viewvc/*checkout*/hurdextras/xmlfs/README The example xml file can be found here: http://cvs.savannah.gnu.org/viewvc/*checkout*/hurdextras/xmlfs/example.xml

xmlfs allows one to create a directory tree out of an xml file. Cool eh?

xmlfs - a translator for accessing XML documents

This is only an alpha version. It works in read only. It supports text nodes and attributes. It doesn't do anything fancy like size computing, though. Here is an example of how to use it:

CDATA, again !

$ cat book0 Mark Twain 4242 $ cat book0/author/.text Mark Twain

An example file, example.xml, is provided. Of course, it does not contain anything useful. xmlfs has been tested on several-megabytes XML documents, though.

Comments are welcome.

-- Manuel Menal

add some info to these store pages:

add some more details to the Hurd libraries pages

is the tmpfs translator broken? 'cause page says it is.

add some more detail to the hurd/gnumach debugging page on the hurd wiki

libports
libstore
libchannel
libtrivfs
libdiskfs
libihash
libpthread
libfshelp
libps

I guess the gsync patches and rpctrace did not play well together. So youpi had to do some magic to get them to play nicely.

rpctrace is basically a man-in-the-middle, and that'll have side effects.

read this . It's got great detail!

I can mention that one can download the old gnuhurd emails, but should mention how to convert to maildir [2/3]

Parallel tracing is completely asyncronous and out of the way.

http://batleth.sapienti-sat.org/projects/mb2md/

mb2md -d .mail/bug-hurd-archive-md

run this command first

mb2md -m -d ~/.mail will tell mb2md where to store the maildirs...

mb2md -s sourcefile -d destdir

take a mbox file and convert it to the destination area.

mb2md -s sourcedir -f somefolder mb2md -s ~/.mail/bug-hurd-archive -f ~/.mail/bug-hurd-archive-md

take a source directory of mboxs and convert it to maildirs

mb2md -s ~/.mail/bug-hurd-archive

produced

~/Maildir/{cur,tmp,new} and

~/Maildir/.{2018-0}{1,2,3,4,5,6,7} each one of these hidden files had many files underneith them.

DONE mentioned how to download the emails

DONE mentioned the tool to convert to maildir

TODO did not show how to convert to maildir

how about a crazy hurd possibilities page?

State "DONE" from [2018-12-13 Thu 11:09]
State "DONE" from [2018-12-13 Thu 11:43]
State "TODO" from [2018-12-13 Thu 11:43]

I can add a page on the Hurd to talk about some of the crazy things that you could do with the Hurd!?

Suppose that you have a large source code. You would like to find all occurancies of "(TODO|FIXME)"

You could do something like this:


settrans -pa /hurd/run  grep -r '(TODO|FIXME)' TODOs

add a more concrete example of the confused deputy problem on the capability page

Now every time that you read the TODOs, it'll grep -r through the whole directory, and point out all of your TODOs and FIXMEs! You never have to run that command again. You can just browse the file. file:///home/joshua/prog/gnu/hurd/hurd-web.rendered/capability.html This is the main page template.

I can get some info from this task list

add an open issues PyHurd page.

add a tutorial showing someone how to run the python, perl, gcc, gdb, glibc, etc. testsuite failures.

add a page showing off the status of the language bindings:

I should also add in CSS on the file. file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/open_issues/some_todo_list.html

sort all of the files in the unsorted directory

fix this on the wiki. It goes no where.

I should add a google summer of code project to make the mach message simplier and more modern

perl
java
python
lisp

I sent an email asking for suggestions. As well as writing some down.

namespace translators I should document that you can (fopen ".,,0/") opens a directory tree with translators disabled!

Or you could mount a translator to unzip a file via "files.zip/gunzip"!

file:///home/joshua/prog/gnu/hurd/hurd-web.rendered/hurd/translator/nsmux.html

apparently the Hurd has a way of opening up a special file, which auto starts a translator!

put some of Neal's paper's on the wiki

I should show off some of the things that it can do.

add to the list of tiny project ideas

http://walfield.org/pub/people/neal/papers/hurd-misc/

create a list of ideas that would be suitable for a thesis!

We could make rm -rf work faster file:///home/joshua/prog/gnu/hurd/hurd-web.rendered/open_issues/rm_fr.html

A new rpctrace new filesystems etc.

Port to other architectures
Persistance
Resource management
Spector/Meltdown mitigation
Setuid vs. constructor
more filesystems!
realtime
Mach performance things?
64 bit port
some cool translator?
sshfs
gitfs

I can apparently change mach's console keymap? That's kind of cool!

https://lists.gnu.org/archive/html/bug-hurd/2019-01/msg00001.html

Perhaps I should add an updated entry about this.

copy/read the arch hurd installation page somewhere. it's got lots of good details.

merge the faq with the old faq and older hurd faq

open_issues/keymap_mach_console.html The debian wiki should be updated, or this information should be stored on the hurd wiki somewhere. https://wiki.archhurd.org/index.php/Installation_Guide#A_note_on_hardware_support

file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/faq.html

file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/faq/old_hurd_faq.html

file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/faq/old_faq.html

Apparently will destroy GNU/Hurd translators. Is this still true?

write some introductory material for the anatomy of a hurd system

file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/open_issues/anatomy_of_a_hurd_system.html

https://teythoon.cryptobitch.de/posts/bootstrapping-the-hurd/

DONE remove items from the open issue page.

State "DONE" from [2020-10-01 Thu 16:19]

explain the Hurd Boot process

add some info to this page

systemd https://www.gnu.org/software/hurd/microkernel/mach/gnumach/interface/syscall.html

It's currently very empty

test that the symlink is still not working

I did add a mach_msg segment. file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/open_issues/symlink_translator.html

add in some Hurd possibilities

Hmm. It does still seem to be broken. use this https://archive.fosdem.org/2018/schedule/event/microkernel_hurd_pci_arbiter/ talk to mention some of the hurd possibilities.

change the savannah repo to not point at an old repo

take a snapshot of the Hurd possibilities to explain the translators.

remove implement xattr support from the open issues page

visudo works just fine, we can remove this webpage https://www.gnu.org/software/hurd/open_issues/visudo.html

the link on this page goes nowhere

delete the wine page?

secure chroot implementation is done?

The savannah repo online points to an empty repository. This should be changed. file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/open_issues/xorg_porting.html

add my youtube channel to the Hurd

http://www.gnu.org/software/hurd/community/gsoc/project_ideas.html

file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/community/facebook.html

do we need gnumig to compile libc?

They mention the facebook channel!

add arch specific commands to start, install, and run the hurd in a VM

there's a webpage that mentions a bridging translator would be cool, but we already have one?

file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/microkernel/mach/mig/gnu_mig.html

https://www.gnu.org/software/hurd/hurd/running/qemu.html#index8h1

https://www.gnu.org/software/hurd/hurd/translator/wishlist.html

copy this page info into the mach 5 proposal

Search for bridg in both of these pages. They contradict?

file:///home/joshua/prog/gnu/hurd/hurd-web.rendered/open_issues/rework_gnumach_ipc_spaces.html

how to boot the Hurd with rumpdisk in qemu:

file:///home/joshua/prog/gnu/hurd/hurd-web.rendered/microkernel/mach/gnumach/projects/mach_5.html

Okay, so rumpdisk "just works"(TM) on latest Debian.

After discussing on IRC

https://logs.guix.gnu.org/hurd/2023-05-18.log#091300

using

https://cdimage.debian.org/cdimage/ports/latest/hurd-i386/debian-hurd-20220824.img

I upgraded to lastest greatest:

--8<---------------cut here---------------start------------->8--- wget http://ftp.de.debian.org/debian/pool/main/d/debian-ports-archive-keyring/debian-ports-archive-keyring_2023.02.01_all.deb dpkg -i debian-ports-archive-keyring_2023.02.01_all.deb apt-get update apt-get dist-upgrade --8<---------------cut here---------------end--------------->8---

changed /boot/grub/grub.cfg

--8<---------------cut here---------------start------------->8--- # multiboot /boot/gnumach-1.8-486.gz root=part:2:device:hd0 console=com0 multiboot /boot/gnumach-1.8-486.gz root=part:2:device:wd0 console=com0 noide --8<---------------cut here---------------end--------------->8---

and /etc/fstab

--8<---------------cut here---------------start------------->8--- #/dev/hd0s2 / ext2 defaults 0 1 dev/wd0s2 ext2 defaults 0 1 #/dev/hd0s1 none swap sw 0 0 /dev/wd0s1 none swap sw 0 0 #/dev/hd2 /media/cdrom0 iso9660 noauto 0 0 /dev/wd2 /media/cdrom0 iso9660 noauto 0 0 --8<---------------cut here---------------end--------------->8---

I got:

Checking root file system.../dev/wd0s2: clean, 42057/259072 files, 416312/1035776 blocks

full log attached.

Now to replicate this on Guix...I already notice that this setup uses acpi, which is broken an has been disabled in the latest rumpkernel build...

Thanks for all the help!

Janneke Nieuwenhuizen | GNU LilyPond https://LilyPond.org Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

document portinfo --search

GNU Hurd / debugging

booting a 64-bit debian ramdisk

From Sergey > I have made changes so that it does daily builds and I'm able to > boot small programs. However, I haven't had the time to boot > programs built against Glibc. How do you package and boot the static > binaries using a ramdisk? I've been reading the other threads about > the Guix/rumpkernel so I might be able to piece something together > and try it this weekend.

You just put the entirety of the root filesystem (containing /usr, /bin, /lib, /hurd, and so on) as an ext2 image into a file that you place onto the actual drive (a CD disk in my case), and then you ask GRUB to load the file from the drive into memory, tell gnumach to make a ramdisk device out of it (you'll need to apply [0]), and tell ext2fs to use that device. Here's the relevant piece of my grub config script:

[0]: https://salsa.debian.org/hurd-team/gnumach/-/blob/master/debian/patches/50_initrd.patch

multiboot /boot/gnumach console=com0 module /boot/initrd.ext2 initrd.ext2 '$(ramdisk-create)' module /sbin/ext2fs.static ext2fs --multiboot-command-line='${kernel-command-line}' --readonly --host-priv-port='${host-port}' --device-master-port='${device-port}' --exec-server-task='${exec-task}' --kernel-task='${kernel-task}' -T device rd0 '$(fs-task=task-create)' '$(prompt-task-resume)' module /lib/ld.so.1 ld.so.1 /hurd/exec --device-master-port='${device-port}' '$(exec-task=task-create)' boot

(I should probably change it to not hardcode 'rd0', but whatever). Note that /boot/gnumach, /boot/initrd.ext2, /sbin/ext2fs.static, and /lib/ld.so.1 are all paths inside the CD image (those are going to be loaded by GRUB), and /boot/initrd.ext2 is the ext2 filesystem image containing the actual Hurd root. /hurd/exec however is already a path inside the fs image -- this is where ld.so (not grub) is going to load the exec server from. The only static binary here is ext2fs.static, the rest are all dynamically linked.

Then in /libexec/console-run (inside the filesystem image), I have written the following:

#! /bin/sh

settrans -ac /dev/mach-console /hurd/streamio console exec <>/dev/mach-console >&0 2>&0 echo Hello from /bin/sh! exec /bin/sh -i

(If you're going to do the same, don't forget to create the /dev/mach-console node beforehand, since the fs is read-only.) I also had to patch streamio a little to do the \r -> \n conversion like glibc already does in devstream:

add in debuggin early init

I've recently been doing this kind of debugging early boot-up process a lot, so maybe I could provide some tips indeed. For getting more lines of output, try console=com0 on gnumach cmdline, and run qemu with -nographic -serial stdio or something like that.

Other than that, just attach gdb and see what it crashes on? Like this:

$ gdb /path/to/gnumach (gdb) tar rem :1234 (gdb) b i386_exception (gdb) b task_terminate (gdb) b Panic (gdb) add-symbol-file /path/to/rumpdisk.static blah-blah (y/n?) y (gdb) c

This is so much easier to do with statically linked non-PIE binaries loaded by gnumach/GRUB at startup compared to hunting for shared library .text addresses and single-stepping through code pages getting paged in upon first access (can't place a breakpoint before the page gets paged in!), so enjoy it while it lasts :)

If you do hit i386_exception, you can look at active_threads[0]->task->name to understand what task it is (though it's likely to be just the rumpdisk in your case). If you step up several frames (perhaps just one), you'll find a 'regs' argument being passed to a function; from there you can extract the faulting %eip, and then can disas around it to see what it is (again, much easier with symbols!).

The trick I like to use is I, upon hitting an exception, re-set all the registers to the values described by 'regs', just like this:

(gdb) set $rsp = $2.uesp (gdb) set $rip = $2.eip ...and so on

(don't forget to switch back to the topmost frame first, with 'down' or 'select-frame') and that basically rewinds time to when the fault has happened, and from there you can see the userland backtrace and inspect the full state at the time of the fault.

running in qemu

getting rumpdisk to work for sata hard drive

Janneke Nieuwenhuizen, le jeu. 18 mai 2023 09:48:11 +0200, a ecrit: > > Try '-M q35' passed to qemu as this will use a newer chipset emulation > > that includes AHCI controller by default. Im not sure if IDE is > > working at this point. > > As this was by far the easiest change, I tried this first. It turns out > that even my debian image (debian-hurd-20220824.img) doesn't boot for me > using this flag (see log below).

part:2:device:hd0: No such device or address

With AHCI your disk is called sd0, not hd0.

Samuel

guix invocation

I've now succesfully been doing

--8<---------------cut here---------------start------------->8--- ./pre-inst-env guix system image -t hurd-raw gnu/system/examples/bare-hurd.tmpl cp /gnu/store/r5dpblnfsj08jh3hdmn8s6l9xaczwn65-disk-image guix.img sudo losetup -o $((512*2048)) /dev/loop0 guix.img sudo mount /dev/loop0 /mnt edit /mnt/boot/grub/grub.cfg, adding console=com0 sudo umount /mnt --8<---------------cut here---------------end--------------->8---

and then

--8<---------------cut here---------------start------------->8--- guix shell qemu -- qemu-system-i386 \ -m 4096 \ --enable-kvm \ --device rtl8139,netdev=net0 \ --netdev user,id=net0,hostfwd=tcp:0.0.0.0:11022-:2222 \ --snapshot \ --no-reboot \ --device virtio-serial-pci \ --nographic \ --serial mon:stdio \ --hda r5dpblnfsj08jh3hdmn8s6l9xaczwn65-disk-image+CONSOLE=COM0 --8<---------------cut here---------------end--------------->8---

Just for fun, find the succesful log attached.

Is there a way to pass the "console=com0" argument from the QEMU command line? That would be nice!

> Other than that, just attach gdb and see what it crashes on? Like this: > > $ gdb /path/to/gnumach > (gdb) tar rem :1234 > (gdb) b i386_exception > (gdb) b task_terminate > (gdb) b Panic > (gdb) add-symbol-file /path/to/rumpdisk.static > blah-blah (y/n?) y > (gdb) c

[..]

disable ide drives aka gnumach drivers and use rumpdisk instead

Ah, I had no idea; this is so helpful. Maybe a good idea to have this on the website/wiki, right? pass "noide" on gnumach cmdline to disable them, or just compile them out. I don't see it in your rumpdisk output, but when run this way it typically discovers that the kernel is already driving IDE, and does nothing.

Then you're passing "hd0s1" to ext2fs as the device to open; that's again a reference to the Mach-implemented device (and partition). The rumpdisk drive is named 'wd0', and you also have to do partitions libstore-side, so: root=part:1:device:wd0

Sergey

delete these pages ?

https://darnassus.sceen.net/~hurd-web/open_issues/glibc_libpthread_robust_mutexes/

https://darnassus.sceen.net/~hurd-web/open_issues/libpthread/

https://darnassus.sceen.net/~hurd-web/open_issues/running_rump_for_slash/

I can test this code:

Does this fail?

https://darnassus.sceen.net/~hurd-web/open_issues/mmap_write-only/

I can move this webpage:

https://darnassus.sceen.net/~hurd-web/hurd/translator/ext2fs/page_cache/

I can update these pages:

https://darnassus.sceen.net/~hurd-web/open_issues/dbus/

with the content from this email:

https://lists.nongnu.org/archive/html/bug-hurd/2013-12/msg00150.html

https://darnassus.sceen.net/~hurd-web/open_issues/problematic_packages/

https://darnassus.sceen.net/~hurd-web/open_issues/emacs/

https://darnassus.sceen.net/~hurd-web/open_issues/kvm/

add in a web page that describes ftpfs

on the exec translator page: hurd/translator/exec

I see a link to the exec_memory issue page. This is no longer a problem. and I deleted that web page. Why is the page still there?

add a wiki page about what to do after running the hurd for the first time

DONE writing a 2018 Q4 qoth

State "DONE" from [2019-01-04 Fri 07:26]

file:///home/joshua/programming/gnu/hurd/hurd-web.rendered/contributing/web_pages/news/qoth_next.html

pci work

merged PCI_Arbiter and should be able to use on Debian GNU/Hurd

http://lists.gnu.org/archive/html/bug-hurd/YYYY-MM/threads.html http://lists.debian.org/debian-hurd/YYYY/MM/ http://lists.gnu.org/archive/html/bug-hurd/2018-10/msg00029.html

making libpciaccess support the hurd

Joan Lledo` updated lwip to work with the latest lwip

Damien Zammit created an ACPI translator

Samuel Thibault allows all users to make tmpfs

Samuel fixed a bfs issue with /proc not being mounted

LLVM now supports the Hurd

porting Rust and Go is ongoing

add file record locking support was this added?

futher polishing by Damien Zammit http://lists.gnu.org/archive/html/bug-hurd/2018-11/msg00033.html AKA only one process can access the PCI config ports https://lists.x.org/archives/xorg-devel/2018-November/057677.html https://lists.x.org/archives/xorg-devel/2018-November/057691.html http://lists.gnu.org/archive/html/bug-hurd/2018-11/msg00062.html http://lists.gnu.org/archive/html/bug-hurd/2018-11/msg00049.html http://lists.gnu.org/archive/html/bug-hurd/2018-11/msg00012.html https://lists.debian.org/debian-hurd/2018/10/msg00007.html https://www.phoronix.com/scan.php?page=news_item&px=GNU-Hurd-LLVM-Clang https://lists.debian.org/debian-hurd/2018/11/msg00020.html http://lists.gnu.org/archive/html/bug-hurd/2018-11/msg00058.html real hardware

https://gist.github.com/AlmuHS/f0c036631881756e817504d28217a910

hurd.org 199 KB Permalink History Raw

Introduction

Documentation things

dde is not really being maintained. These are drivers from the linux kernel. The Rump kernel seems to be a better option.

Important programs to use

settrans

showtrans

fsysopts

Vocab

Port things

Port

Port Bucket

Port Class

atomic operations vs. global locks

File

Capabilities things

Capabilities

Capability operating system

Principle of least privilege

ABAC

File Descriptor

Translator

Active Translator

Passive Translator

Short Circuiting

Translator problems

TODO Translators set up by untrusted users

Trust the behavior of Translators

Translator pages I should read:

https://www.gnu.org/software/hurd/hurd/translator/writing/example.html

Setting up translators - HowTo

TODO I can give more cool examples with translators

https://www.gnu.org/software/hurd/hurd/translator.html

https://www.gnu.org/software/hurd/hurd/documentation/translators.html

https://www.gnu.org/software/hurd/hurd/debugging/translator.html

protid

Virtual File System

Computer Bought the Farm

Identity Based Access Control

Access Control Lists

real time things

real time operating system

realtime

locks

mcs locks

ticket locks

page

stack

heap

Hurd

Things the Hurd can do

SATA drives

The Hurd now supports 128GB partition sizes

types of virtualization

subhurds

remap one service for another

Hurd Servers

To get help about how to use a server run

Auth

remap

process server

Users can have more than one uid

password

pfinet

symlink

the hurd servers parse arguments with argp. I can learn more about it here:

crash server

Hurd translators

hello

xmlfs

stowfs

run

unionfs

Hurd Libraries

Trivfs

Real basic trivfs stuff

various functions and variables that must be defined

Trivfs has some options. Read about it on p. 35 of the hurd reference manual.

libnetfs

libdiskfs

hurd.org 199 KB

Permalink History Raw