Tutorial Series

This is Part 3 of a tutorial series. Also see:

• Part#1 - My take on SimpleHTTPServer in go
• Part#2 - Trying to achieve code quality
• Part#3 - I can haz featurez?
• Part#4 - Eyecandy, anyone?

Summary

In this part I want to implement some features. Those are:

• Basic Auth
• Self-Signed Certificate support
• Provide own Certificate support
• File Upload

Right now we are still using only standard libraries so far.

Preparation

To be able to implement our new Features we will need to alter main.go and internal/myhttp/fileserver.go a bit. The new FileServer Struct has to hold a bunch of new parameters.

package main

import (
	"flag"
	"os"

	"github.com/patrickhener/goshs/internal/myhttp"
)

var (
	port       = 8000
	webroot    = "."
	ssl        = false
	selfsigned = false
	myKey      = ""
	myCert     = ""
	basicAuth  = ""
)

func init() {
	wd, _ := os.Getwd()

	// flags
	flag.IntVar(&port, "p", port, "The port")
	flag.StringVar(&webroot, "d", wd, "Web root directory")
	flag.BoolVar(&ssl, "s", ssl, "Use self-signed TLS")
	flag.BoolVar(&selfsigned, "ss", selfsigned, "Use self-signed certificate")
	flag.StringVar(&myKey, "sk", myKey, "Path to own server key")
	flag.StringVar(&myCert, "sc", myCert, "Path to own server cert")
	flag.StringVar(&basicAuth, "P", basicAuth, "Use basic auth password (user: gopher)")

	flag.Parse()
}

func main() {
	// Setup the custom file server
	server := &myhttp.FileServer{
		Port:       port,
		Webroot:    webroot,
		SSL:        ssl,
		SelfSigned: selfsigned,
		MyCert:     myCert,
		MyKey:      myKey,
		BasicAuth:  basicAuth,
	}
	server.Start()
}

As you can see I defined a lot of new flags and default values to hold our chosen parameters. Also the server “object” is now instantiated with all those flags. For this to work we need to prepare the FileServer struct within internal/myhttp/fileserver.go to reflect this parameters like so:

// FileServer holds the fileserver information
type FileServer struct {
	Port       int
	Webroot    string
	SSL        bool
	SelfSigned bool
	MyKey      string
	MyCert     string
	BasicAuth  string
}

With this setup we can later on reference everything we need to know with fs.something.

Basic Authentication

To be able to restrict access to our file server provided by goshs we can leverage basic authentication to secure our service. For this purpose we are writing two new functions for our fileserver.

// authRouter will hook up the webroot with the fileserver using basic auth
func (fs *FileServer) authRouter() {
	http.HandleFunc("/", fs.basicAuth(fs.ServeHTTP))
}

// basicAuth is a wrapper to handle the basic auth
func (fs *FileServer) basicAuth(handler http.HandlerFunc) func(w http.ResponseWriter, req *http.Request) {
	return func(w http.ResponseWriter, req *http.Request) {
		w.Header().Set("WWW-Authenticate", `Basic realm="Restricted"`)

		username, password, authOK := req.BasicAuth()
		if authOK == false {
			http.Error(w, "Not authorized", http.StatusUnauthorized)
			return
		}

		if username != "gopher" || password != fs.BasicAuth {
			http.Error(w, "Not authorized", http.StatusUnauthorized)
			return
		}

		fs.ServeHTTP(w, req)
	}
}

The first function is a replacement routing function for the previous defined function router. So if the user chooses to have basic authentication instead of using router() when starting the server we are using authRouter(). We will take care of this in a second.

The second function is a Basic Authentication wrapper around our handler function. It will be used in the authRouter() function to be wrapped around fs.ServeHTTP which itself will trigger the handler function. Fortunately the builtin http.Request has a BasicAuth() method already waiting for us to be used to check for the validity of the request and to parse the username and password.

First of all we set the Header WWW-Authenticate. This will trigger the browser to request credentials if none are provided. Then we grab the basic auth credentials from our request and check if it is a valid basic authentication. If so we check for the username and password. The user in this part is hard-coded to be gopher whereas the password is provided by the flag -P and stored in our file servers parameter BasicAuth if you can recall from the beginning.

If the credentials match we server the content via fs.ServerHTTP(). Otherwise the browser again will be requesting for valid credentials.

With this code our server is ready to handle Basic Authentication.

Self-Signed Certificate

So, this part was not so tricky afterall. I have to admit that the code was taken from this blog post by Shane Utt and then altered to my liking. Therefore most of the credits go to Shane Utt.

I created a new package internal/myca/ca.go to hold all of the Certificate Authority stuff for me. So what it basically does is to define a Setup() function to generate a Certificate Authority and afterwards generate a server certificate to be used by http.ListenAndServeTLS(). I hard-coded the values of the server certificate and the Certificate Authority within the code. One could provide enough flags to enable the user to choose his own certificate values. But I decided on hard-coding it for simplicity.

The complete file can be seen in the github repo tag v.0.0.3.

I extended the Setup() function a little bit. What I did is to write a function which will parse the certificate generated and then return the sha256 and sha1 fingerprint. I wanted to be able to display the fingerprints to the user on the servers console. As the certificate is self-signed the user now is able to check for the validity of the certificate himself.

// Sum will give the sha256 and sha1 sum of the certificate
func Sum(cert []byte) (sha256s, sha1s string) {
	// Building sha256 sum
	var f256 [32]byte
	f256 = sha256.Sum256(cert)
	sha256s = fmt.Sprintf("%X", f256)

	b := strings.Builder{}
	b.Grow(len(sha256s) + len(sha256s)/2 + 1)

	for i := 0; i < len(sha256s); i++ {
		b.WriteByte(sha256s[i])
		if i%2 == 1 {
			b.WriteByte(' ')
		}
	}

	sha256s = b.String()

	// building sha1 sum
	var f1 [20]byte
	f1 = sha1.Sum(cert)
	sha1s = fmt.Sprintf("%X", f1)

	b = strings.Builder{}
	b.Grow(len(sha1s) + len(sha1s)/2 + 1)

	for i := 0; i < len(sha1s); i++ {
		b.WriteByte(sha1s[i])
		if i%2 == 1 {
			b.WriteByte(' ')
		}
	}

	sha1s = b.String()

	return sha256s, sha1s

}

As you can see the function will take the certificate in []byte format and then do a simple sha256.Sum256 and a sha1.Sum over it. This will generate the fingerprint we know when looking at the certificate warning in our browsers.

With a so called string builder I am formatting the returned fingerprint into the format the browser will display which is 2 characters of the hashsum devided by whitespaces and all uppercase like so:

❯ go run main.go -s -ss
2020/10/13 13:22:43 Serving HTTP on 0.0.0.0 port 8000 from /tmp/goshs with ssl enabled and self-signed certificate
2020/10/13 13:22:43 WARNING! Be sure to check the fingerprint of certificate
2020/10/13 13:22:43 SHA-256 Fingerprint: BF E4 E7 74 69 8F 81 C5 E9 88 EE C2 4F 39 C6 60 1B 64 FD AA 40 34 97 46 F1 BC 75 BF 3D 46 39 09
2020/10/13 13:22:43 SHA-1   Fingerprint: C3 CE 90 39 7E FA 12 00 3A C7 35 EE 28 8A 8B E5 C8 1C E5 15

So now it is possible to check if the connection is secured and the certificate is correct.

Provide own certificate

As I was playing around with the code a bit further the idea grew to me that someone might want to provide a certificate himself instead of letting goshs generate one. Well this is also quite easy.

So image someone doing this:

❯ openssl ecparam -genkey -name secp384r1 -out server.key
❯ openssl req -new -x509 -sha256 -key server.key -out server.crt -days 3650
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:DE
State or Province Name (full name) [Some-State]:BW
Locality Name (eg, city) []:hesec.de
Organization Name (eg, company) [Internet Widgits Pty Ltd]:SimpleHTTPServer
Organizational Unit Name (eg, section) []:hesec.de
Common Name (e.g. server FQDN or YOUR name) []:127.0.0.1:8000
Email Address []:gopher@hesec.de

This will result in two files server.key and server.crt. We will integrate the possibility to provide those to http.ListenAndServerTLS() in a second. But to be also able to check on the fingerprints of such a certificate I added another function to parse the server.crt and then be able to return the hashsums:

// ParseAndSum will take the user provided cert and return the sha256 and sha1 sum
func ParseAndSum(cert string) (sha256s, sha1s string, err error) {
	certBytes, err := ioutil.ReadFile(cert)
	if err != nil {
		return "", "", err
	}

	block, _ := pem.Decode(certBytes)

	certParsed, err := x509.ParseCertificate(block.Bytes)
	if err != nil {
		return "", "", err
	}

	sha256s, sha1s = Sum(certParsed.Raw)

	return sha256s, sha1s, nil
}

This function will use x509.ParseCertificate() of the standard library to transform the pem decoded certificate into []byte form. Then I can use the output to feed it to the previously defined Sum() function and return the fingerprints.

Upload a File

I also coded another useful feature. The user should be able to not only download files from the server but to push a file, as well. So I coded an upload() handler extending our file server:

// upload handles the POST request to upload files
func (fs *FileServer) upload(w http.ResponseWriter, req *http.Request) {
	req.ParseMultipartForm(10 << 20)

	file, handler, err := req.FormFile("file")
	if err != nil {
		log.Printf("Error retrieving the file: %+v\n", err)
	}
	defer file.Close()

	// Get url so you can extract Headline and title
	upath := req.URL.Path

	// construct target path
	targetpath := strings.Split(upath, "/")
	targetpath = targetpath[:len(targetpath)-1]
	target := strings.Join(targetpath, "/")

	// Construct absolute savepath
	savepath := fmt.Sprintf("%s%s/%s", fs.Webroot, target, handler.Filename)

	// Create file to write to
	if _, err := os.Create(savepath); err != nil {
		log.Println("ERROR:   Not able to create file on disk")
		fs.handle500(w, req)
	}

	// Read file from post body
	fileBytes, err := ioutil.ReadAll(file)
	if err != nil {
		log.Println("ERROR:   Not able to read file from request")
		fs.handle500(w, req)
	}

	// Write file to disk
	if err := ioutil.WriteFile(savepath, fileBytes, os.ModePerm); err != nil {
		log.Println("ERROR:   Not able to write file to disk")
		fs.handle500(w, req)
	}

	// Log request
	mylog.LogRequest(req.RemoteAddr, req.Method, req.URL.Path, req.Proto, "200")

	// Redirect back from where we came from
	http.Redirect(w, req, target, http.StatusSeeOther)
}

First I parse the multipart form which will be displayed to the user by the template (we will do that in a second). Then I create a file handler by parsing the forms field with the name “file”. Doing this I will have the possibility to get the filename for example.

Afterwards I construct the target path which has to be the folder I am currently in plus the relative path my browser points to plus the filename of the uploaded file.

Then I simply create the file (which is like touch <filename.extension> in linux) and then copy the bytes of the uploaded file into the newly created file. Also I am logging the events to the console and handle the errors if any occure.

Finally I redirect the user to the path where he came from to reload the page. So the newly uploaded file will be displayed immediately.

The template for the index page has to be changed as well to provide the html multipart form:

const dispTmpl = `
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    <title>Directory listing for {{.Path}}</title>
  </head>
  <body>
    <div>
    <h1>Upload File</h1>
    {{ if (eq .Path "/") }}
    <form id="upload" name="upload" enctype="multipart/form-data" autocomplete="off" action="/upload" method="POST">
    {{ else }}
    <form id="upload" name="upload" enctype="multipart/form-data" autocomplete="off" action="{{.Path}}/upload" method="POST">
    {{ end }}
    <input name="file" type="file" id="upload" />
    <input type="submit" value="upload">
    </form>
    </div>
    <hr />
    <div>
    <h1>Directory listing for {{.Path}}</h1>
    <hr />
	<ul>
	  {{range .Content}}
		<li><a href="/{{.URI}}">{{.Name}}</a></li>
	  {{ end }}
  </ul>
  </div>
    <hr />
  </body>
</html>
`

As you can see I extended the dispTmpl with a form. Also I leveraged a if-else statement of the templating engine to handle an error which occured when uploading to the browsers root (/) path.

So to distinguish between our upload request and a simple dir listing request I chose to implement a switch statement within fileserver.go. I changed the function ServerHTTP() of our fileserver to be like this:

// ServeHTTP will serve the response by leveraging our handler
func (fs *FileServer) ServeHTTP(w http.ResponseWriter, req *http.Request) {
	defer func() {
		if err := recover(); err != nil {
			http.Error(w, fmt.Sprintf("%+v", err), http.StatusInternalServerError)
		}
	}()

	switch req.Method {
	case "GET":
		fs.handler(w, req)
	case "POST":
		fs.upload(w, req)
	}
}

Whenever a GET request hits the fileserver the function fs.handler() will be called. But when a POST request occures the fs.upload() handler will handle the request. The form described above will have POST as a method. This way we can distinguish between an upload request and a dir listing request to the same route.

Final part

To make all of our new features work we need to integrate them when starting our file server. I rewrote the fs.Start() function to look like this:

// Start will start the file server
func (fs *FileServer) Start() {
	// init router with or without auth
	if fs.BasicAuth != "" {
		if !fs.SSL {
			log.Printf("WARNING!: You are using basic auth without SSL. Your credentials will be transfered in cleartext. Consider using -s, too.\n")
		}
		log.Printf("Using 'gopher:%+v' as basic auth\n", fs.BasicAuth)
		fs.authRouter()
	} else {
		fs.router()
	}

	// construct server
	add := fmt.Sprintf(":%+v", fs.Port)
	server := http.Server{Addr: add}

	// Check if ssl
	if fs.SSL {
		// Check if selfsigned
		if fs.SelfSigned {
			serverTLSConf, fingerprint256, fingerprint1, err := myca.Setup()
			if err != nil {
				log.Fatalf("Unable to start SSL enabled server: %+v\n", err)
			}
			server.TLSConfig = serverTLSConf
			log.Printf("Serving HTTP on 0.0.0.0 port %+v from %+v with ssl enabled and self-signed certificate\n", fs.Port, fs.Webroot)
			log.Println("WARNING! Be sure to check the fingerprint of certificate")
			log.Printf("SHA-256 Fingerprint: %+v\n", fingerprint256)
			log.Printf("SHA-1   Fingerprint: %+v\n", fingerprint1)
			log.Panic(server.ListenAndServeTLS("", ""))
		} else {
			if fs.MyCert == "" || fs.MyKey == "" {
				log.Fatalln("You need to provide server.key and server.crt if -s and not -ss")
			}

			fingerprint256, fingerprint1, err := myca.ParseAndSum(fs.MyCert)
			if err != nil {
				log.Fatalf("Unable to start SSL enabled server: %+v\n", err)
			}

			log.Printf("Serving HTTP on 0.0.0.0 port %+v from %+v with ssl enabled server key: %+v, server cert: %+v\n", fs.Port, fs.Webroot, fs.MyKey, fs.MyCert)
			log.Println("INFO! You provided a certificate and might want to check the fingerprint nonetheless")
			log.Printf("SHA-256 Fingerprint: %+v\n", fingerprint256)
			log.Printf("SHA-1   Fingerprint: %+v\n", fingerprint1)

			log.Panic(server.ListenAndServeTLS(fs.MyCert, fs.MyKey))
		}
	} else {
		log.Printf("Serving HTTP on 0.0.0.0 port %+v from %+v\n", fs.Port, fs.Webroot)
		log.Panic(server.ListenAndServe())
	}
}

Basic Auth

First of all when starting the file server I check if basic authentication was chosen by the user. If so, but the user forgot to choose TLS as well I warn him for transfering the credentials in cleartext. This is an issue but I do not restrict the app beeing used like this.

Afterwards the router is setup with fs.authRouter(). If no basic auth is chosen the router is setup with fs.router() as before.

Then I construct a new http.Server object. This will make it possible to handle all the cases given by the combinations of possible flags. First I add the address to it by setting the Attribute Addr. Then I check if the user requested to have TLS.

TLS Case: Self-Signed

If the user chose to have a self-signed certificate I call my myca.Setup() function and retrieve the servers TLS configuration, as well as the fingerprint hashes. Then I add the TLS Configuration to the instantiated http.Server und start it by calling server.ListenAndServeTLS("", ""). Usually ListenAndServeTLS will want to have a path to the servers key and the servers certificate. But as we already defined a TLSConfig with a valid certificate and assigned it to the server this will just work.

TLS Case: User provided certificate

In case the user provided both a certificate and a key I will use my myca.ParseAndSum() function to retrieve the fingerprint hashes and then start the server with the user provided certificate. The two certificate functions are very similar but as you can see when calling this variant of TLS I use server.ListenAndServeTLS(fs.MyCert, fs.MyKey) to hand over the two paths to the certificate and the key the user provided.

Conclusion

Now our goshs is able to upload files, be secured with basic auth and use TLS either with a self-signed certificate or with a user provided one. As always feedback is very welcome in the comment section below and I hope you had a good time reading. The final code of this tutorial can be found as tag v0.0.3 in my github repository.

Thanks for reading,

Patrick