Hi,
I know that with MirageOS it is possible to send HTTP requests.
However, I want to utilise the Slack postMessage API which establishes a HTTPS connection.
I need to run this on Solo5 and when I do, I get the following error message:
Fatal error: exception (Failure “connect: TLS is not supported”)
Cheers
Fatal error: exception (Failure “connect: TLS is not supported”)
Currently, the way to be able to handle HTTPS as client into a MirageOS is a bit hidden by underlying constraints but I suspect that a dependence to tls-mirage into your config.ml will probably fix the error - it will probably update conduit to integrate a TLS connection as a possible way to communicate with a peer.
However, I’m currently on this kind of problem where I started (just yesterday) to update conduit to something else than what we have - a more explicit and extensible implementation of it. You can see details here: https://github.com/mirage/ocaml-conduit/pull/311
Right,
Is this what you mean (in config.ml):
...
let packages = [ package "packageA" ; .... ; package "tls-mirage";]
...
?
If so then this didn’t work unfortunately 
hello,
the error stems from the conduit-mirage library. I’m not sure how your setup is exactly, but there are two ways to fix it:
(a) in config.ml you likely have a conduit_direct stack – if you change this to conduit_direct ~tls:true stack, it should work
(b) in unikernel.ml where you have the conduit module and value, you can let conduit = CONDUIT.with_tls conduit in
The sad side of the story is that atm OCaml-TLS won’t work with slack (AFAICT) https://github.com/ocurrent/ocurrent/pull/174 – but there’ll be (really soon now) TLS 1.3 support for OCaml-TLS (https://github.com/mirleft/ocaml-tls/pull/405), which should solve that issue.
You can give the TLS 1.3 support a try (to work as a solo5 unikernel, you’ll need to opam pin add fiat-p256 --dev and opam pin add hacl_x25519 --dev, next to opam pin add tls https://github.com/hannesm/ocaml-tls.git#tls13 and opam pin add tls-mirage https://github.com/hannesm/ocaml-tls.git#tls13). I’m interested to get your feedback 
FWIW the slack docs indicate they work with TLS 1.2 + SNI
So I followed step (a) in config.ml.
For step (b), Conduit is only used/referred to in the module definition and nowhere else:
module Client (T: Mirage_time.S) (C: Mirage_console.S) (RES: Resolver_lwt.S) (CON: Conduit_mirage.S) = struct
I also tried using TLS 1.3.
All of these attempts resulted in the same error message:
Fatal error: exception (Failure "connect: HANDSHAKE_FAILURE")
You can find the config.ml and and unikernel.ml files in my github repo.
Hope this helps
in the link you provided, the unikernel connects to http://192.168.0.37:7379/RPOP/sensor_data which is neither https nor a slack endpoint. According to your error HANDSHAKE_FAILURE, it looks like TLS is enabled in your unikernel, but there’s an issue while connecting.
For (b), there’s the module CON of module type Conduit_mirage.S and your start function has a ctx : CON.targument, thus let ctx_with_ssl = CON.with_tls ctx in ... would be needed.
I meant that there are two options, (a) and (b) - and one is sufficient, since (a) will tell the mirage utility to inject the code from (b) (in the main.ml module which is generated by mirage).
The Unikernel makes 2 connections, the first one “http://192.168…” works fine as this is something else. It is the second connection:
“https://slack.com/api/chat.postMessage?token=ENTER-SLACK-TOKEN&” which fails.
See here.
Cheers
hello,
a slightly modified unikernel:
$ diff -wur slackbla.orig slackbla
diff -wur slackbla.orig/config.ml slackbla/config.ml
--- slackbla.orig/config.ml 2020-04-26 13:30:07.126698000 +0200
+++ slackbla/config.ml 2020-04-26 13:18:50.017950000 +0200
@@ -14,6 +14,6 @@
let () =
let stack = generic_stackv4 default_network in
let res_dns = resolver_dns stack in
- let conduit = conduit_direct stack in
+ let conduit = conduit_direct ~tls:true stack in
let job = [ client $ default_time $ default_console $ res_dns $ conduit ] in
register "sck-alert" job
diff -wur slackbla.orig/unikernel.ml slackbla/unikernel.ml
--- slackbla.orig/unikernel.ml 2020-04-26 13:30:25.557085000 +0200
+++ slackbla/unikernel.ml 2020-04-26 13:28:28.666763000 +0200
@@ -27,28 +27,29 @@
let http_fetch c resolver ctx uri_pop uri_set=
let const_ctx = ctx in
let ctx = Cohttp_mirage.Client.ctx resolver ctx in
- Cohttp_mirage.Client.get ~ctx uri_pop >>= fun (response, body) ->
+(* Cohttp_mirage.Client.get ~ctx uri_pop >>= fun (response, body) ->
Cohttp_lwt.Body.to_string body >>= fun body ->
let json = Yojson.Basic.from_string body in
let open Yojson.Basic.Util in
let value = json |> member "RPOP" |> to_string in
let thresh = threshold value 50.0 in
- if thresh then
- let uri_set = Uri.of_string (String.concat uri_set ["";(String.concat (String.concat "text=ALERT-" ["";value]) ["";"&pretty=1"])]) in
+ if thresh then *)
+ (* let uri_set = Uri.of_string (String.concat uri_set ["";(String.concat (String.concat "text=ALERT-" ["";value]) ["";"&pretty=1"])]) in *)
+ let uri_set = Uri.of_string uri_set in
Cohttp_mirage.Client.get ~ctx uri_set >>= fun (res, body) ->
+ Logs.app (fun m -> m "received reply %s"
+ (Cohttp.Code.string_of_status res.Cohttp.Response.status));
+ Cohttp_lwt.Body.to_string body >>= fun data ->
+ Logs.app (fun m -> m "received body %s" data);
Lwt.return_unit
- else
- Lwt.return_unit
+(* else
+ Lwt.return_unit *)
(* output (threshold value 50.0) uri_set ctx c *)
-
-
-
let start _time c res (ctx:CON.t) =
let ns = Key_gen.resolver ()
and uri_get = Uri.of_string "http://192.168.0.37:7379/RPOP/sensor_data"
and uri_message = "https://slack.com/api/chat.postMessage?token=ENTER-SLACK-TOKEN&"
in
http_fetch c res ctx uri_get uri_message
-
end
and a set of pins (as mentioned above):
$ opam pin
fiat-p256.0.2.0 git git+https://github.com/mirage/fiat.git
hacl_x25519.0.1.0 git git+https://github.com/mirage/hacl.git
tls.0.11.1 git git+https://github.com/hannesm/ocaml-tls#tls13
tls-mirage.0.11.1 git git+https://github.com/hannesm/ocaml-tls#tls13
is able to successfully establish a HTTPS connection to slack.com (after mirage configure -t hvt && mirage build).
here’s the output of a run:
$ doas solo5-hvt --net:service=tap1 -- sck_alert.hvt --ipv4=10.0.42.2/24 --ipv4-gateway=10.0.42.1
| ___|
__| _ \ | _ \ __ \
\__ \ ( | | ( | ) |
____/\___/ _|\___/____/
Solo5: Bindings version v0.6.4
Solo5: Memory map: 512 MB addressable:
Solo5: reserved @ (0x0 - 0xfffff)
Solo5: text @ (0x100000 - 0x427fff)
Solo5: rodata @ (0x428000 - 0x4a8fff)
Solo5: data @ (0x4a9000 - 0x6a4fff)
Solo5: heap >= 0x6a5000 < stack < 0x20000000
2020-04-26 11:28:39 -00:00: INF [netif] Plugging into service with mac 5a:af:a4:88:45:bd mtu 1500
2020-04-26 11:28:39 -00:00: INF [ethernet] Connected Ethernet interface 5a:af:a4:88:45:bd
2020-04-26 11:28:39 -00:00: INF [ARP] Sending gratuitous ARP for 10.0.42.2 (5a:af:a4:88:45:bd)
2020-04-26 11:28:39 -00:00: INF [udp] UDP interface connected on 10.0.42.2
2020-04-26 11:28:39 -00:00: INF [tcpip-stack-direct] stack assembled: mac=5a:af:a4:88:45:bd,ip=10.0.42.2
2020-04-26 11:28:39 -00:00: INF [application] here
2020-04-26 11:28:39 -00:00: APP [application] received reply 200 OK
2020-04-26 11:28:39 -00:00: APP [application] received body {"ok":false,"error":"invalid_auth"}
Solo5: solo5_exit(0) called
without the pins (i.e. using the released tls version), it leads to a handshake failure.
All the best,
hannes
Cheers for the help.
For some reason though I can’t compile this now:
Error: The files ~/.opam/4.10.0/lib/conduit-mirage/conduit_mirage.cmi and unikernel.cmi make inconsistent assumptions over interface Tls__Packet
What version of software are you using?
I have:
opam --version 2.0.0
ocaml --version The OCaml toplevel, version 4.10.0
mirage --version v3.7.6
Also, I have found that when adding the pins, I am no longer able to build ANY unikernels and get the following errors:
~/.opam/4.10.0/lib/hacl_x25519/freestanding/libhacl_x25519_freestanding_stubs.a(Hacl_Curve25519.o): In function
Hacl_EC_Format_fexpand': ~/.opam/4.10.0/.opam-switch/build/hacl_x25519.0.1.0/_build/default/freestanding/src/Hacl_Curve25519.c:658: undefined reference tole64toh’
~/.opam/4.10.0/.opam-switch/build/hacl_x25519.0.1.0/_build/default/freestanding/src/Hacl_Curve25519.c:660: undefined reference tole64toh' ~/.opam/4.10.0/.opam-switch/build/hacl_x25519.0.1.0/_build/default/freestanding/src/Hacl_Curve25519.c:662: undefined reference tole64toh’
~/.opam/4.10.0/.opam-switch/build/hacl_x25519.0.1.0/_build/default/freestanding/src/Hacl_Curve25519.c:664: undefined reference tole64toh' ~/.opam/4.10.0/.opam-switch/build/hacl_x25519.0.1.0/_build/default/freestanding/src/Hacl_Curve25519.c:666: undefined reference tole64toh’
~/.opam/4.10.0/lib/hacl_x25519/freestanding/libhacl_x25519_freestanding_stubs.a(Hacl_Curve25519.o): In functionHacl_EC_Format_fcontract_store': ~/.opam/4.10.0/.opam-switch/build/hacl_x25519.0.1.0/_build/default/freestanding/src/Hacl_Curve25519.c:781: undefined reference tohtole64’
~/.opam/4.10.0/.opam-switch/build/hacl_x25519.0.1.0/_build/default/freestanding/src/Hacl_Curve25519.c:782: undefined reference tohtole64' ~/.opam/4.10.0/.opam-switch/build/hacl_x25519.0.1.0/_build/default/freestanding/src/Hacl_Curve25519.c:783: undefined reference tohtole64’
~/.opam/4.10.0/.opam-switch/build/hacl_x25519.0.1.0/_build/default/freestanding/src/Hacl_Curve25519.c:784: undefined reference to `htole64’
Thanks
Error: The files ~/.opam/4.10.0/lib/conduit-mirage/conduit_mirage.cmi and unikernel.cmi make inconsistent assumptions over interface Tls__Packet
This looks like you should run a mirage clean in the unikernel directory first (and then mirage configure && mirage build).
undefined reference tole64toh’
undefined reference tohtole64’
About the undefined reference to htole64 – did you pin hacl_x25519 and fiat-p256 to the development repositories? I’m using OCaml 4.10 and mirage 3.7.6 as well – and a clang 9.0.1 as C compiler (on a FreeBSD host) – it may be that your C compiler / freestanding toolchain does not include a htole64.
We should work on fixing this before releasing the TLS 1.3 stack.