This thesis summarizes my work on using ultrafast laser pulses to study Terahertz (THz) electrodynamics of photonic and electronic nanostructures and microstructures. Ultrafast time-resolved (optical, NIR, MIR, THz) pump-probe spectroscopy setup has been successfully built, which enables me to perform a series of relevant experiments. Firstly, a novel high efficiency and compact THz wave emitter based on split-ring-resonators has been developed and characterized. The emitter can be pumped at any wavelength by tailoring the magnetic resonance and could generate gapless THz waves covering the entire THz band. Secondly, two kinds of new photonic structures for THz wave manipulation have been successfully designed and characterized. One is based on the 1D and 2D photo-imprinted diffractive elements. The other is based on the photoexcited double-split-ring-resonator metamaterials. Both structures are flexible and can modulate THz waves with large tunability. Thirdly, the dark excitons in semiconducting single-walled carbon nanotubes are studied by optical pump and THz probe spectroscopy, which provides the first insights into the THz responses of nonequilibrium excitonic correlations and dynamics from the dark ground states in carbon nanotubes. Next, several on-going projects are briefly presented such as the study of ultrafast THz dynamics of Dirac fermions in topological insulator Bi$_{2}$Se$_{3}$ with Mid-infrared excitation. Finally, the thesis ends with a summary of the completed experiments and an outlook of the future plan.