So over the last month I've been very busy helping out Hackaday in preparation for the announcement of The Hackaday Prize.

During the lead up to the announcement we wanted to build some suspense at Hackaday so we started releasing a series of space themed puzzles over all over April. Every week we posted a new puzzle, starting with Transmission #01 which was released on April 1st.

These went down very well and in all we had a few hundred people participate in trying to solve them, much fun was had and an amazing display of ingenuity, persistence and downright evil genius was shown by the Hackaday community at large.

Since we've been asked a fair number of questions about why we did this and how we put it together I've started a series of articles over at Hackaday documenting the event. You can checkout the first on Hackaday Space : Transmission 1 now.

]]>The question is: How to design a good transform on the signal we want to analyze? It becomes a classical problem in signal representation- We want to define a transform which provides a "sparse" representation of the signal which capture most or all information of a signal. "Sparseness" is one of the reasons for the extensive use of popular transforms, because they discover the structure of the singal and provide a "compact" representation. In this post, we provide an example that how to analyze the web traffic by Discrete Fourier Transform (DFT).

In every hour, we record the total number of user actions in our website and the data is shown in figure 1. This signal seems to be periodic, but the question is: How could we explore the structure? DFT provides a nice tool to represent the discrete time signal into periodic Fourier series. The original signal can be represented by

\begin{equation}

\label{eq:1}

x(t_n) = \frac{1}{\sqrt{N}} \sum_{k=0}^{N-1} X_k \cdot e^{i 2 \pi k n / N},

\quad n\in\mathbb{Z}\

\end{equation}

where the coefficients of Fourier series are defined as

\begin{equation}

\label{eq:2}

X_k = \frac{1}{\sqrt{N}} \sum_{n=0}^{N-1} x(t_n) \cdot e^{-i 2 \pi k n / N}, \quad k\in\mathbb{Z}\

\end{equation}

After DFT, we can analyze the energy in each frequency component. The frequency component is given by , where is the sampling frequency. We plot the power spectrum of the data in figure 2. The result shows the web traffic is mostly on low-frequency components.

Let's look at different frequency components of the data: We compare different frequency components with the original data: the original signal is shown in figure 3(a), the components with frequency is in figure 3(b), and components with frequency in figure 3(c). Note that the traffic in weekdays is much higher than weekend, and the first components successfully capture the traffic change between weekdays and weekend. Also, the traffic is much higher for working hours than the rest periods, which can be clearly identified for the daily change in the second components.

Based on sparse representation, we can select ``significant'' components for many different applications, i.e, signal de-noising and or pattern recognition. We will introduce the wavelet transform in the next blog.

]]>Our latest product, Datasheet.net, is now finally available for anyone to use. After a few months in private beta we feel ready to let the world see what we've been playing with. Here's a quote from the going live post on the Datasheet.net blog.

We think datasheets suck. Plain and simple this is a format that hasn’t changed since the invention of the PDF. We’re trying to change that, trying to make datasheets a ‘live’ document, something that actually helps make our lives as engineers better. In the long term this means us working closely with manufacturers to help improve the datasheet format, we’re not sure where this leads yet but its something we’re working towards.

But changing the way manufacturers work is a long and slow road, so in the meantime as a first step we’ve put together some useful features on top of datasheets as they currently stand.

We'll be promoting the project more over the coming weeks, and hopefully extending it to support all manner of new features and cool things. But for now signup to Datasheet.net to start making datasheets a little bit better!

]]>Big Data Camp LA 2013 will be the key event of the season bringing different sides of this community together. SupplyFrame is one of the sponsors and a bunch of our engineers will be attending. We have our fair share of challenges, experiences and ideas in this space and if you're the same way, we would love to chat ! That's what unconferences are for.

See you at ~~#~~**BigDataCampLA** !

I'm really looking forward to getting my grubby paws on one of these awesome conference badges and hacking up some fun projects while we're there.

Check out the other sponsors of the event, and get yourself some tickets before they run out!

]]>