In this paper, we consider a mobile-edge computing (MEC) system, where an access point (AP) assists a mobile device (MD) to execute an application consisting of multiple tasks following a general task call graph. The objective is to jointly determine the offloading decision of each task and the resource allocation (e.g., CPU computing power) under time-varying wireless fading channels and stochastic edge computing capability, so that the energy-time cost (ETC) of the MD is minimized. Solving the problem is particularly hard due to the combinatorial offloading decisions and the strong coupling among task executions under the general dependency model. To address the issue, we propose a deep reinforcement learning (DRL) framework based on the actor-critic learning structure. In particular, the actor network utilizes a deep neural network (DNN) to learn the optimal mapping from the input states (i.e., wireless channel gains and edge CPU frequency) to the binary offloading decision of each task. Meanwhile, for the critic network, we show that given the offloading decision, the remaining resource allocation problem becomes convex, where we can quickly evaluate the ETC performance of the offloading decisions output by the actor network. Accordingly, we select the best offloading action and store the state-action pair in an experience replay memory as the training dataset to continuously improve the action generation DNN. Numerical results show that for various types of task graphs, the proposed algorithm achieves up to 99.5% of the optimal performance while significantly reducing the computational complexity compared to the existing optimization methods.