ANSWERED (bash) attempting to direct output of goldcrest to named pipe in script

[luxc0r0 0]

EDIT: Simplified example:

 

mkfifo /tmp/test
goldcrest --air-connect | tee /tmp/test

Also tried:

goldcrest --air-connect 2>&1 | tee /tmp/test

In neither case am I able to view or capture the output of goldcrest command in its entirety – it displays normally up until about a dozen lines after "EVENT: CONNECTING" and then freezes. See example output log below

Original:

Hoping someone who knows more about how goldcrest outputs its data might help me solve this puzzle! I had made a bash script like this in the past that I used with eddie-cli. I am attempting to re-write it for goldcrest. I hope to complete it and share it

The script is in my bin directory, so it is intended to be run outside of any interactive terminal. In my case, it is launched via dmenu.

The script opens a terminal in which goldcrest --air-connect is automatically run; it uses two named pipes so that the usual output can still appear in the terminal while another part of the screen reads all the output line by line, triggering WM notification scripts when a line matches "EVENT: CONNECTING", "EVENT: CONNECTED", or "EVENT: DISCONNECTED".

I might change change this triggers, and will add other features like automatically minimizing the system tray to the systray, etc.  

The problem is not much output of goldcrest --air-connect displays in the terminal after  "EVENT: CONNECTING". Consistently, about a dozen more line print, and then the output freezes 3 characters into the year on the timestamp, always in the same place, printing "202" for "2022". In reality, goldcrest successfully connects; if I wait several minutes, the output in the terminal will eventually update, and my notifications will run as well. Likewise, only notification that ever runs is the one tied to "CONNECTING". If I hit Ctrl+C, the output unfreezes, and I receive my connected and disconnected signals simultaneously.

These are the two lines I suspect I need to modify, and I've monkeyed around with them a lot with no luck.

tee -i "$_gc_pipe" < "$_gc_int" &
goldcrest --air-connect > "$_gc_int" 

The pipes are indeed created and they are being read before they are printed to. Everything technically sort of works if you wait a very long time; the output is not like normal.

Here are the last lines that display, showing the exact place where it freezes:
 

2022-06-27 23:22:01 EVENT: CONNECTING
2022-06-27 23:22:01 Tunnel Options:V4,dev-type tun,link-mtu 1522,tun-mtu 1500,proto UDPv4,comp-lzo,cipher AES-256-GCM,auth [null-digest],keysize 256,key-method 2,tls-client
2022-06-27 23:22:01 Peer Info:
IV_VER=3.6.6 AirVPN
IV_PLAT=linux
IV_NCP=2
IV_TCPNL=1
IV_PROTO=30
IV_CIPHERS=AES-256-GCM
IV_LZO_STUB=1
IV_COMP_STUB=1
IV_COMP_STUBv2=1
UV_IPV6=no
IV_GUI_VER=Bluetit - AirVPN OpenVPN 3 Service 1.0.0
IV_SSL=OpenSSL 1.1.0l  10 Sep 2019

2022-06-27 23:22:01 VERIFY OK: depth=1, /C=IT/ST=IT/L=Perugia/O=airvpn.org/CN=airvpn.org CA/emailAddress=info@airvpn.org, signature: RSA-SHA512
202

Here is the script in its (undeveloped) entirety:
 

#!/usr/bin/env bash
#luxc0r0

_gc_term_title="Goldcrest Interface"
_gc_systray_icon="$HOME/Icons/Mine/goldcrest.png"
_gc_notify_icon="$HOME/Icons/Mine/goldcrest_tr.png"
_gc_notify_sound="$HOME/Audio/scifisounds/PremiumBeat_0013_cursor_click_11.wav"
_gc_notify_timeout="10"
_gc_tmp_dir='/mnt/ramdisk'

_gc_notify() {
	"$HOME/scripts/./naughty" "$_gc_term_title" "$1" "$2" "$_gc_notify_icon" "$_gc_notify_timeout" & 2>/dev/null
    [[ -f "$_gc_notify_sound" ]] && paplay "$_gc_notify_sound" &
}

_gc_popup_wrapper() { setsid st -c GoldCrest -t "$_gc_term_title" -e bash -c "$0 --child; bash" ; }
#_gc_popup_wrapper() { setsid xfce4-terminal -T "$_gc_term_title" -e "bash -c \"$0 --child; bash\"" ; }


_gc_pipe="${_gc_tmp_dir}/.goldcrest_fifo1"
_gc_int="${_gc_tmp_dir}/.goldcrest_fifo2"

if [[ "$1" == "--child" ]] ; then


    #launch notification daemon if not already running
	[[ "$(ps -aux | grep "$0 --notif[y]")" ]] || { "$0" "--notify" & >/dev/null; }


    tee -i "$_gc_pipe" < "$_gc_int" &
    goldcrest --air-connect > "$_gc_int" 

    echo "${0}: Exiting."
    sleep 2

elif [[ "$1" == "--notify" ]] ; then

    #for reference: manually kill notification daemon if needed
    #kill -15 "$(ps -aux | grep -i "crestPop --notif[y]" | awk '{print $2}')"

    [[ ! -p "$_gc_pipe" ]] && mkfifo "$_gc_pipe"
    [[ ! -p "$_gc_int" ]] && mkfifo "$_gc_int"

    while true
    do
        while read info
        do
            # if echo "$info" | grep "Connected to AirVPN server" >/dev/null ; then
            #     _gc_server=$(echo "$info" | sed 's/[0-9:\-]//g; s/^  //' )
            # fi

            if echo "$info" | grep "EVENT: CONNECTING" >/dev/null ; then

                _gc_notify "Connecting..." 

            elif echo "$info" | grep "EVENT: CONNECTED" >/dev/null ; then

                # [[ -n "$_gc_server" ]] && _gc_notify "$_gc_server" "#aa9900"
                # [[ -z "$_gc_server" ]] && _gc_notify "Connected! " "#aa9900"

                _gc_notify "Connected!" "#aa9900"

            elif echo "$info" | grep "EVENT: DISCONNECTED" >/dev/null ; then
                _gc_notify "Disconnected." 
            fi
        done < "$_gc_pipe"
    done

else

    _gc_popup_wrapper

fi

I notice that there is a lot of output from goldcrest and I wonder if that causes problems, although it doesn't seem like my system is freezing. I wonder if there is a different operator I should use for the redirects. Curious if anyone happens to see this and understand it! I hope I am just overlooking something obvious 
  Edited ... by luxc0r0

[luxc0r0 0]

The main thread does not need to be so complicated. For my purposes I can simply use one pipe; however, the ability even to do that seems compromised. Is there no way to reliably/consistently redirect the output of goldcrest --air-connect to a file/named pipe?  

[coenzyme 0]

On 6/28/2022 at 2:37 PM, luxc0r0 said:

mkfifo /tmp/test
goldcrest --air-connect | tee /tmp/test

Also tried:

goldcrest --air-connect 2>&1 | tee /tmp/test

15 hours ago, luxc0r0 said:

Is there no way to reliably/consistently redirect the output of goldcrest --air-connect to a file/named pipe?

I don't use goldcrest so can't confirm, but can the following link solve your problem?

1. Process Substitution (Bash Reference Manual)
2. shell - How to duplicate a stream and process both parts in a streaming way? - Unix & Linux Stack Exchange

[luxc0r0 0]

9 hours ago, coenzyme said:

I don't use goldcrest so can't confirm, but can the following link solve your problem?

1. Process Substitution (Bash Reference Manual)
2. shell - How to duplicate a stream and process both parts in a streaming way? - Unix & Linux Stack Exchange

Thanks for the effort. It appears though that any method to redirect the stream of goldcrest's output works for a few dozen lines and then gets stuck right around the moment it connects.

[luxc0r0 0]

Here's the most recent thing I tried:

goldcrest --air-connect | tee >(grep EVENT) > "$pipe"

The idea was that maybe the pipe was getting "overloaded" (somehow) and that the output should be filtered (via grep) before it's sent to the file, but the pipe still freezes in the same place, i.e. on the "CONNECTING" hook. 

[Staff 9755]

On 7/6/2022 at 2:36 PM, luxc0r0 said:

Here's the most recent thing I tried:

goldcrest --air-connect | tee >(grep EVENT) > "$pipe"

The idea was that maybe the pipe was getting "overloaded" (somehow) and that the output should be filtered (via grep) before it's sent to the file, but the pipe still freezes in the same place, i.e. on the "CONNECTING" hook. 

Hello!

The problem is related to tee when it processes buffered fast output (Goldcrest and Bluetit are very fast). See also: https://askubuntu.com/questions/639877/tee-doesnt-get-whole-output-from-the-pipe

By changing the initial standard out stream to "unbuffered" you will resolve the problem. Example:

stdbuf -o0 ./goldcrest --air-connect | tee /tmpfs/gc.log

Kind regards
 

[luxc0r0 0]

On 7/8/2022 at 11:36 AM, Staff said:

Hello!

The problem is related to tee when it processes buffered fast output (Goldcrest and Bluetit are very fast). See also: https://askubuntu.com/questions/639877/tee-doesnt-get-whole-output-from-the-pipe

By changing the initial standard out stream to "unbuffered" you will resolve the problem. Example:

stdbuf -o0 ./goldcrest --air-connect | tee /tmpfs/gc.log

 Kind regards
 

You are amazing! Thank you