Hi,

I’m running into a weird bug with http_client. If I try to write the response to a file, it crashes out. Here’s the callback:

static void get_message_response_cb(struct http_response *rsp,
                        enum http_final_call final_data,
                        void *user_data)
{
	ssize_t output = 0;
    int ret;

    if (rsp->body_found) 
    {
        output = fs_write(&file, rsp->body_frag_start, rsp->body_frag_len);
        //output = fs_write(&file, "hello", 5);
        if (output <= 0) {
            LOG_INF("output: %d", output);
        }
    }
}

The weird thing is that the program will log output: 0, but write data to the file anyway, but just once! Then there will be just one subsequent call to the callback function, which will have http status of 0.

If I comment out the first call to fs_write and uncomment the second, then print the fill contents out, it will have hello in there several times, which is expected since the file I’m downloading is longish.

Any idea what’s going on here?

Here’s the function that kicks off the request in case its helpful:

int get_message(char* file_name)
{
    int fd = -1;
    int ret = 0;
    uint8_t recv_buf_ipv4[BUFFER_SIZE] = {0};

    // open file for recording response
	fs_file_t_init(&file);
	ret = fs_open(&file, file_name, FS_O_CREATE | FS_O_APPEND);
	if (ret < 0)
	{
		LOG_ERR("FAIL: open %s: %d", file_name, ret);
	} 

    LOG_INF("Getting Message From: %s", CONFIG_CLOUD_HOSTNAME);

    /* Setup socket */
    fd = socket_setup();
    if (fd < 0)
    {
        LOG_ERR("Unable to setup socket. Err: %i", ret);
        return ret;
    }

    LOG_INF("Socket setup complete");

    struct http_request req;
    memset(&req, 0, sizeof(req));
    //MAYBE? change to old header?
    char *const headers[] = {
        //"Transfer-Encoding: chunked\r\n",
        "Connection: close\r\n",
        NULL
    };

    char * payload = "empty";

    req.method = HTTP_GET;
    req.url = "/api/messages/1/1/500";
    req.host = CONFIG_CLOUD_HOSTNAME;
    req.protocol = "HTTP/1.1";
    req.payload = payload;
    req.payload_len = strlen(payload);
    req.header_fields = (const char **)headers;
    req.response = get_message_response_cb;
    req.recv_buf = recv_buf_ipv4;
    req.recv_buf_len = sizeof(recv_buf_ipv4);
    //req.content_type_value = "application/json";

    ret = http_client_req(fd, &req, timeout, NULL);
    if (ret < 0)
        LOG_ERR("Unable to send data to cloud. Err: %i", ret);

    /* Close connection */
    (void)close(fd);
    LOG_INF("closing file");
    fs_close(&file);

    return 0;
}

thank you!
Jason

I tried adjusting the heap and stack sizes, too, btw!

@theIntuitionist do you have the console output? I’m trying to understand the issue.

You need to be careful you only access the FS from the same thread. It’s also not designed for frequent read/writes.

Here is the console output:

*** Booting Zephyr OS build v3.3.99-ncs1-2 ***
[00:00:00.252,960] <inf> main: Init storage
[00:00:00.252,990] <inf> littlefs: LittleFS version 2.5, disk version 2.0
[00:00:00.253,997] <inf> littlefs: FS at w25q32jv@0:0x0 is 1024 0x1000-byte blocks with 512 cycle
[00:00:00.254,028] <inf> littlefs: sizes: rd 16 ; pr 16 ; ca 64 ; la 32
[00:00:00.575,836] <inf> littlefs_storage: bsize = 16 ; frsize = 4096 ; blocks = 1024 ; bfree = 989
[00:00:00.632,781] <inf> main: Init modem
[00:00:00.868,438] <inf> main: Init Network
[00:00:00.918,060] <inf> network: Certificate match
[00:00:00.921,447] <inf> main: Connect to LTE Network
[00:00:06.134,948] <inf> main: File removed
[00:00:06.156,768] <inf> web: Getting Message From: okthatworks.com
[00:00:06.156,799] <inf> web: Looking up okthatworks.com
[00:00:08.302,612] <inf> web: Connected!
[00:00:08.302,642] <inf> web: Socket setup complete
[00:00:09.351,531] <inf> web: output: 1
[00:00:09.351,715] <inf> web: output: 20
[00:00:09.351,837] <inf> web: output: 20
[00:00:09.351,959] <inf> web: output: 20
[00:00:09.680,541] <inf> web: output: 20
[00:00:09.680,633] <inf> web: output: 0
[00:00:09.681,793] <inf> web: closing file
[00:00:09.682,861] <inf> main: 
[00:00:09.704,376] <inf> littlefs_storage: 123456789012345678901234567890123456789012345678901234567890123
[00:00:09.704,589] <inf> littlefs_storage: 456789012345678901

that’s from a run where I made the receive buffer 20 char. It made it through a few reads, which is interesting, not just a single read, then died before enough callbacks to write the whole file. The last two littlefs_storage lines are me just printing the contents of the file.

How do I think about “It’s also not designed for frequent read/writes”? The core functionality of the device I’m building is recording audio on one device, uploading it to a server where a second device then downloads it and plays it. This may happen a few to several times a day. Reading i2s audio, uploading and downloading audio all happens in chunks, so functionally there are infrequent bursts of read/write activity.

thank you for your help!
Jason

The really confounding part is that if I just write the string “hello” to persistent memory instead of the data coming in from the http response, things work fine every time. That makes me think it isn’t hardware limitations- though, in all honestly, I’m totally stumped!

If you’re writing to the file in get_message_response_cb you may very well be in an interrupt context which is not allowed in Zephyr. I suggest using a message queue or ring buffer to move the received data over to another thread for processing/FS work. Since the FS is slow, you don’t want to block that code since it will cause problems across your application.

I typically have a data thread I create that does all the FS work. I use a message queue to push data into the thread.

void data_thread(void *, void *, void *)
{

    int res;

    for (;;)
    {
        struct data_event evt = {0};
        size_t encoded_size = 0;

        res = k_msgq_get(&event_msq, &evt, K_FOREVER);
        if (res < 0)
        {
            LOG_WRN("Unable fetch from message queue. Err: %i", res);
            continue;
        }

       // TODO: do something
}

#define DATA_STACK_SIZE KB(12)
K_THREAD_DEFINE(data_tid, DATA_STACK_SIZE,
                data_thread, NULL, NULL, NULL,
                K_LOWEST_APPLICATION_THREAD_PRIO, 0, 0);

ok! thank you. I will give this a shot.

Terms and Conditions | Privacy Policy