Quantcast
Channel: ClearScript
Viewing all 2297 articles
Browse latest View live

Commented Issue: [BUG] Script execution blocks V8 script item finalization [86]

$
0
0
The V8 script item finalizer currently requires a V8 runtime lock. If the runtime is executing a long-running script, it can block the finalization thread and cause out-of-control memory consumption and frequent garbage collection.
Comments: I just added a discussion item that I think could be related to this. https://clearscript.codeplex.com/discussions/642428

Commented Unassigned: Exception when calling ToString() on Primitive Object [87]

$
0
0
I am trying to call ToString() on an int object and I am getting the following exception

An unhandled exception of type 'Microsoft.ClearScript.ScriptEngineException' occurred in ClearScriptV8-32.dll

Additional information: TypeError: obj.IntTest.ToString is not a function.

Below is a snippet of code which is causing the issue.

__DataObject.cs__

```
using System;
using System.Collections.Generic;

namespace T4EditorTest
{
public class DataObject
{
public DataObject()
{
Children = new List<SubDataObject>();
}

public Guid Id { get; set; }

public string SystemName { get; set; }

public string DisplayName { get; set; }

public string Description { get; set; }

public Char CharTest { get; set; }

public int IntTest { get; set; }

public List<SubDataObject> Children { get; set; }

public string IntTestToString()
{
return IntTest.ToString();
}
}

public class SubDataObject
{
public DataObject Parent { get; set; }

public Guid Id { get; set; }

public string SystemName { get; set; }

public string DisplayName { get; set; }

public string Description { get; set; }
}
}
```
__Calling Code__
```
private static void TestScriptEngine()
{
using (var engine = new V8ScriptEngine())
{
engine.AddHostType(typeof(DataObject));
engine.AddHostType("Console", typeof(Console));

const string script = @"
var obj = new DataObject();
obj.DisplayName = 'Hello, Script Engine.';
Console.WriteLine(obj.DisplayName);
//obj.CharTest = '1';
obj.IntTest = 5;
obj.SystemName = obj.IntTest.ToString();
Console.WriteLine(obj.IntTest);";

engine.Execute(script);

var obj = engine.Script["obj"] as DataObject;
}
}
```
Comments: All the reported behavior is by design.

Closed Unassigned: Exception when calling ToString() on Primitive Object [87]

$
0
0
I am trying to call ToString() on an int object and I am getting the following exception

An unhandled exception of type 'Microsoft.ClearScript.ScriptEngineException' occurred in ClearScriptV8-32.dll

Additional information: TypeError: obj.IntTest.ToString is not a function.

Below is a snippet of code which is causing the issue.

__DataObject.cs__

```
using System;
using System.Collections.Generic;

namespace T4EditorTest
{
public class DataObject
{
public DataObject()
{
Children = new List<SubDataObject>();
}

public Guid Id { get; set; }

public string SystemName { get; set; }

public string DisplayName { get; set; }

public string Description { get; set; }

public Char CharTest { get; set; }

public int IntTest { get; set; }

public List<SubDataObject> Children { get; set; }

public string IntTestToString()
{
return IntTest.ToString();
}
}

public class SubDataObject
{
public DataObject Parent { get; set; }

public Guid Id { get; set; }

public string SystemName { get; set; }

public string DisplayName { get; set; }

public string Description { get; set; }
}
}
```
__Calling Code__
```
private static void TestScriptEngine()
{
using (var engine = new V8ScriptEngine())
{
engine.AddHostType(typeof(DataObject));
engine.AddHostType("Console", typeof(Console));

const string script = @"
var obj = new DataObject();
obj.DisplayName = 'Hello, Script Engine.';
Console.WriteLine(obj.DisplayName);
//obj.CharTest = '1';
obj.IntTest = 5;
obj.SystemName = obj.IntTest.ToString();
Console.WriteLine(obj.IntTest);";

engine.Execute(script);

var obj = engine.Script["obj"] as DataObject;
}
}
```

Edited Issue: Exception when calling ToString() on Primitive Object [87]

$
0
0
I am trying to call ToString() on an int object and I am getting the following exception

An unhandled exception of type 'Microsoft.ClearScript.ScriptEngineException' occurred in ClearScriptV8-32.dll

Additional information: TypeError: obj.IntTest.ToString is not a function.

Below is a snippet of code which is causing the issue.

__DataObject.cs__

```
using System;
using System.Collections.Generic;

namespace T4EditorTest
{
public class DataObject
{
public DataObject()
{
Children = new List<SubDataObject>();
}

public Guid Id { get; set; }

public string SystemName { get; set; }

public string DisplayName { get; set; }

public string Description { get; set; }

public Char CharTest { get; set; }

public int IntTest { get; set; }

public List<SubDataObject> Children { get; set; }

public string IntTestToString()
{
return IntTest.ToString();
}
}

public class SubDataObject
{
public DataObject Parent { get; set; }

public Guid Id { get; set; }

public string SystemName { get; set; }

public string DisplayName { get; set; }

public string Description { get; set; }
}
}
```
__Calling Code__
```
private static void TestScriptEngine()
{
using (var engine = new V8ScriptEngine())
{
engine.AddHostType(typeof(DataObject));
engine.AddHostType("Console", typeof(Console));

const string script = @"
var obj = new DataObject();
obj.DisplayName = 'Hello, Script Engine.';
Console.WriteLine(obj.DisplayName);
//obj.CharTest = '1';
obj.IntTest = 5;
obj.SystemName = obj.IntTest.ToString();
Console.WriteLine(obj.IntTest);";

engine.Execute(script);

var obj = engine.Script["obj"] as DataObject;
}
}
```

Created Issue: [BUG] Weak handle contexts are leaked during explicit disposal of cached V8 objects [90]

$
0
0
When cached V8 objects are explicitly disposed during V8 context destruction, the weak states of their handles must be cleared and the associated weak callback contexts deleted.

New Post: MemoryLeak despite disposing of V8 Script Engines

$
0
0
Hello infinitiesloop!

You've discovered a very subtle bug that's unrelated to Issue #86. We're already testing fixes for both. Thank you!

The excessive thread usage you're seeing is also unrelated, and we cannot reproduce it. Do you have any relevant stacks?

Thanks again!

New Post: MemoryLeak despite disposing of V8 Script Engines

$
0
0
I'm very interested in those fixes! I appreciate how active you guys are on this project still. We're using it for some compelling things :)

It may be that my next move is to wait for your fixes and see where I go from there. If it helps though, I do see some threads that look like this:
OS Thread Id: 0xf260 (17)
Current frame: ntdll!NtWaitForSingleObject+0xa
Child-SP         RetAddr          Caller, Callee
000000002c1bfd10 000007fefd2410dc KERNELBASE!WaitForSingleObjectEx+0x9c, calling ntdll!NtWaitForSingleObject
000000002c1bfd90 000000007712413d ntdll!RtlAllocateHeap+0xd9d, calling ntdll!RtlAllocateHeap+0xdb0
000000002c1bfdb0 000007fee3b46319 *** WARNING: Unable to verify checksum for v8-x64.dll
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for v8-x64.dll - 
v8_x64!v8::ResourceConstraints::code_range_size+0xeb9, calling v8_x64!v8::ArrayBuffer::Contents::Contents+0x26a0
000000002c1bfe40 000007fee3d5f970 v8_x64!v8::ArrayBuffer::Contents::Contents+0x1550
000000002c1bfe70 000007fee8164f7f *** ERROR: Symbol file could not be found.  Defaulted to export symbols for msvcr120.dll - 
msvcr120!beginthreadex+0x107
000000002c1bfea0 000007fee8165126 msvcr120!endthreadex+0x192, calling msvcr120!beginthreadex+0xf0
000000002c1bfed0 0000000076fc652d kernel32!BaseThreadInitThunk+0xd
000000002c1bff00 00000000770fc521 ntdll!RtlUserThreadStart+0x21
So some thread sitting on a Wait within v8-x64.dll. There are a bunch that look like this.

I will continue to play around see if I can come up with a simple reproduction of the run away threads, but not too hopeful I can do it at the moment.

New Post: MemoryLeak despite disposing of V8 Script Engines

$
0
0
Here's the callstack with arguments. It's strange to me that NtWaitForSingleObject is called with all 0s. It's also strange to me that code_range_size is waiting on anything, because in the V8 code, this is just a field lookup:

https://v8docs.nodesource.com/io.js-1.2/d4/da0/v8_8h_source.html#l04338
0:000> ~113 kb
RetAddr           : Args to Child                                                           : Call Site
000007fe`fda210dc : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!NtWaitForSingleObject+0xa
000007fe`c6906319 : 00000000`69160df0 00000000`00000000 00000000`00000000 00000000`000010cc : KERNELBASE!WaitForSingleObjectEx+0x9c
000007fe`c6b1f970 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : v8_x64!v8::ResourceConstraints::code_range_size+0xeb9
000007fe`f2b74f7f : 00000000`69274840 00000000`00000000 00000000`00000000 00000000`00000000 : v8_x64!v8::ArrayBuffer::Contents::Contents+0x1550
000007fe`f2b75126 : 000007fe`f2c2cb80 00000000`00000000 00000000`00000000 00000000`00000000 : msvcr120!beginthreadex+0x107
00000000`77a459cd : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : msvcr120!endthreadex+0x192
00000000`77c7b981 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0xd
00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x21
Any thoughts?

New Post: MemoryLeak despite disposing of V8 Script Engines

$
0
0
Hi again,

Hmm, that stack doesn't look correct at all. As you mentioned, v8::ResourceConstraints::code_range_size() doesn't call anything, and neither does the v8::ArrayBuffer::Contents constructor. Are you sure you have the right symbols?

Cheers!

New Post: MemoryLeak despite disposing of V8 Script Engines

$
0
0
I don't think it is right, but I'm not sure how to get the correct symbols. Here's the 'noisy' output from windbg. I added a MS and Chrome symbol path:
000007fe`fda210dc : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!NtWaitForSingleObject+0xa
SYMSRV:  d:\symbols\v8-x64.pdb\AEFF2A3642C44ACE9BB070E1CFCFCC8D1\v8-x64.pdb not found
SYMSRV:  https://msdl.microsoft.com/download/symbols/v8-x64.pdb/AEFF2A3642C44ACE9BB070E1CFCFCC8D1/v8-x64.pdb not found
SYMSRV:  d:\symbols\v8-x64.pdb\AEFF2A3642C44ACE9BB070E1CFCFCC8D1\v8-x64.pdb not found
SYMSRV:  https://chromium-browser-symsrv.commondatastorage.googleapis.com/v8-x64.pdb/AEFF2A3642C44ACE9BB070E1CFCFCC8D1/v8-x64.pdb not found
DBGHELP: C:\Projects\!OPEN_SOURCE\clearscript\ClearScript\V8\V8\build\v8-x64\build\Release\v8-x64.pdb - file not found
*** WARNING: Unable to verify checksum for v8-x64.dll
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for v8-x64.dll - 
DBGHELP: v8_x64 - export symbols
Where does v8-x64 come from?

New Post: MemoryLeak despite disposing of V8 Script Engines

$
0
0
V8 symbols are generated when you build ClearScript. If you obtained ClearScript via NuGet or another binary package, you could try searching for symbols somewhere within the package directory.

By the way, what ClearScript version are you using?

New Post: MemoryLeak despite disposing of V8 Script Engines

$
0
0
I am using 5.4.2.1, from Nuget. It does not ship with the symbols.

I've tried to build from source, but upon opening the solution, VS complains that it is using an older version of the VC++ compiler/libraries. It tries to update them, but then the build still fails due to missing "v120" tools. Perhaps I just don't have an old enough VS/.NET SDK? Or is there a way for me to just grab the symbols from somewhere?

New Post: MemoryLeak despite disposing of V8 Script Engines

$
0
0
ClearScript's build procedure is outlined in the ReadMe. It currently requires Visual Studio 2013 (which includes the "v120" toolset), but the next point release, with the fixes mentioned above, will support Visual Studio 2015 as well.

New Post: MemoryLeak despite disposing of V8 Script Engines

$
0
0
Got it! Here's a much better call stack. After a few thousand iterations in my system (which equates to a few thousand events being raised -- e.g. js callbacks being called), I end up with almost 200 threads stuck at the same place:
0:000> ~197 kb
RetAddr           : Args to Child                                                           : Call Site
000007fe`fda210dc : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!NtWaitForSingleObject+0xa
000007fe`c59e6989 : 00000000`747e3880 00000000`00000000 00000000`00000000 00000000`000006e0 : KERNELBASE!WaitForSingleObjectEx+0x79
000007fe`c5bffc20 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : v8_x64!v8::internal::OptimizingCompilerThread::Run+0x59 [d:\git\clearscript\clearscript\v8\v8\build\v8-x64\src\optimizing-compiler-thread.cc @ 132]
000007fe`f2b74f7f : 00000000`1e35c9d0 00000000`00000000 00000000`00000000 00000000`00000000 : v8_x64!v8::base::ThreadEntry+0x20 [d:\git\clearscript\clearscript\v8\v8\build\v8-x64\src\base\platform\platform-win32.cc @ 1296]
000007fe`f2b75126 : 000007fe`f2c2cb80 00000000`00000000 00000000`00000000 00000000`00000000 : msvcr120!_callthreadstartex+0x17 [f:\dd\vctools\crt\crtw32\startup\threadex.c @ 376]
00000000`77a459cd : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : msvcr120!_threadstartex+0x102 [f:\dd\vctools\crt\crtw32\startup\threadex.c @ 354]
00000000`77c7b981 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0xd
00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x1d

New Post: MemoryLeak despite disposing of V8 Script Engines

$
0
0
It appears that V8 maintains one "optimizing compiler thread" per isolate - an implementation detail beyond ClearScript's control. If your application uses the V8ScriptEngine constructor, it creates one isolate per engine instance.

You mentioned that you're seeing roughly 200 compiler threads. Is it possible that your application at that point is holding that many engine instances? Our normal recommendation is to use roughly as many concurrent engine instances as there are CPU cores in the machine.

New Post: MemoryLeak despite disposing of V8 Script Engines

$
0
0
I have at most 10-15 scripts active at a time. For each script, I create a separate V8Runtime and create an engine off of that. I compile the script, too. So for each script I have a Tuple<V8Runtime, V8Engine, V8Script>.

At first I did not create an explicit runtime and used the V8Engine constructor. But in an effort to solve this problem I added the explicit V8Runtime so that I could dispose of it explicitly.

So now, after a particular script receives 250 callbacks, I will completely dispose of the script, engine, and runtime, then build them back up again. I was hoping whatever was leaking would get cleaned up that way, but it didn't make any difference.

I could instead keep a pool of V8Runtimes and reuse them and see if that helps. But if I do that, I have a question. These are all long-running scripts, because they are waiting for a callback initiated by managed code. But I want each script to be isolated and not collide any globals between them. I also would like them to execute in parallel if possible -- because each one really has nothing to do with any of the others, so serializing execution of them would be silly and a possible bottleneck for me. So, the question is, if two V8Engines share the same V8Runtime, will they still be isolated from each other in that regard?

New Post: MemoryLeak despite disposing of V8 Script Engines

$
0
0
TL;DR: I may have identified my problem, which was my fault, but I'm not sure if it is expected behavior from ClearScript.

Alright I figured out why my thread count was continuously rising. You saying that it creates that optimization thread per isolate was what helped me find it.

Sometimes my javascript will call into a managed object and expect back a javascript array (I always try to avoid exposing .NET details to the scripts). On the managed side, I have a List<T>. It converts it into an Array by a ClearScript Evaluate("[]") call, then pushes each item into it. Well, I did that wrong. The Evaluate("[]") was being done on a NEW ScriptEngine rather than the Engine from the calling code. So the result was I was returning a js array created from one ScriptEngine back into code from another ScriptEngine.

Apparently when you do that, the object won't be cleaned up until disposing of the receiving engine.

I fixed that, and it's been a huge improvement. I'm still monitoring it for leaks though -- I'm not yet convinced I'm out of the woods. Some of that may just be the known problems you mentioned.

I'm also a bit confused by the results, because my app is very explicitly disposing of the script, engine, and runtime after every 200 calls. It even calls GC.Collect. But the thread count was still rising forever. This sample app sort of reproduces by problem, but I can't reproduce the inability to explicitly clean it up by disposing of things. Perhaps there's yet more subtlety to my app preventing that.

Sample app:
        private const string _script = @"
project.onIssueExceptionStatsChanged(function(ev) {
    var things = ev.getThings();
});
";

        static void Main(string[] args)
        {
            var runtime = new V8Runtime();
            var engine = runtime.CreateScriptEngine();
            var compiledScript = runtime.Compile(_script);

            var host = new HostObject();
            engine.AddHostObject("project", host);
            
            engine.Execute(compiledScript);

            for (var i = 0; i < 10; i++)
            {
                var iteration = i;

                Console.WriteLine("Iteration {0}. Threads = {1}. Memory = {2}.",
                    iteration,
                    Process.GetCurrentProcess().Threads.Count,
                    Process.GetCurrentProcess().PrivateMemorySize64.ToString("n0"));

                for (var j = 0; j < 100; j++)
                {
                    host.RaiseEvent(new SomeEventArgs());
                }
            }

            GC.Collect();
            GC.WaitForPendingFinalizers();

            Console.WriteLine("Done. Hit enter to Dispose. Threads: {0}. Memory: {1}",
                Process.GetCurrentProcess().Threads.Count,
                Process.GetCurrentProcess().PrivateMemorySize64.ToString("n0"));
            Console.ReadLine();


            compiledScript.Dispose();
            engine.Dispose();
            runtime.Dispose();

            // need to do this explicitly or it may not get cleaned up yet...
            GC.Collect();
            GC.WaitForPendingFinalizers();

            Console.WriteLine("Done. Threads: {0}. Memory: {1}",
                Process.GetCurrentProcess().Threads.Count,
                Process.GetCurrentProcess().PrivateMemorySize64.ToString("n0"));
            Console.ReadLine();
        }
    }

    public class HostObject 
    {
        private dynamic _callback;

        public void onIssueExceptionStatsChanged(dynamic callback)
        {
            _callback = callback;
        }

        internal void RaiseEvent(SomeEventArgs args)
        {
            _callback(args);
        }
    }

    public class SomeEventArgs
    {
        public object getThings()
        {
            dynamic list = new V8ScriptEngine().Evaluate("[]");
            list.push("foo");
            return list;
        }
    }
Output:
Iteration 0. Threads = 5. Memory = 30,679,040.
Iteration 1. Threads = 105. Memory = 942,821,376.
Iteration 2. Threads = 205. Memory = 1,850,576,896.
Iteration 3. Threads = 305. Memory = 2,758,254,592.
Iteration 4. Threads = 405. Memory = 3,662,749,696.
Iteration 5. Threads = 505. Memory = 4,570,488,832.
Iteration 6. Threads = 605. Memory = 5,477,806,080.
Iteration 7. Threads = 705. Memory = 6,386,348,032.
Iteration 8. Threads = 808. Memory = 7,291,899,904.
Iteration 9. Threads = 908. Memory = 8,198,541,312.
Done. Hit enter to Dispose. Threads: 1008. Memory: 9,104,228,352

Done. Threads: 4. Memory: 38,354,944

New Post: MemoryLeak despite disposing of V8 Script Engines

$
0
0
Hi infinitiesloop,

The Evaluate("[]") was being done on a NEW ScriptEngine rather than the Engine from the calling code. So the result was I was returning a js array created from one ScriptEngine back into code from another ScriptEngine.

Ah, that explains it. Although ClearScript supports exposing foreign script objects, we don't recommend it. One of the problems is that the receiving engine has no way to discover the "weight" of an external object; it only knows the size of its local proxy. That can lead to suboptimal garbage collection behavior. In this case the external object is very large (as it holds another script engine), whereas the proxy is tiny, so the engine doesn't prioritize its garbage collection.

Apparently when you do that, the object won't be cleaned up until disposing of the receiving engine.

That's correct as long as the receiving engine continues to use the proxy. If it garbage-collects the proxy, the object becomes available for managed garbage collection.

I can't reproduce the inability to explicitly clean it up by disposing of things. Perhaps there's yet more subtlety to my app preventing that.

Please let us know what you find. By the way, the next ClearScript release will use a newer V8 version that seems to have done away with the compiler thread. Here's what your test program's output will look like:
Iteration 0. Threads = 3. Memory = 23,080,960.
Iteration 1. Threads = 3. Memory = 545,955,840.
Iteration 2. Threads = 3. Memory = 1,064,095,744.
Iteration 3. Threads = 3. Memory = 1,583,087,616.
Iteration 4. Threads = 3. Memory = 2,100,662,272.
Iteration 5. Threads = 3. Memory = 2,618,945,536.
Iteration 6. Threads = 3. Memory = 3,135,750,144.
Iteration 7. Threads = 5. Memory = 3,653,799,936.
Iteration 8. Threads = 5. Memory = 4,169,682,944.
Iteration 9. Threads = 5. Memory = 4,688,252,928.
Done. Hit enter to Dispose. Threads: 5. Memory: 5,203,578,880

Done. Threads: 3. Memory: 29,364,224
Cheers!

New Post: Debug script

$
0
0
Hi,

Script loaded dynamically does not hit break point.

I have script ex:
var a=10;
var b=20
var c=a+b;
ScriptLib.LoadScript("sample.js");
var d=a+b+c;
and in LoadScript() function I am reading this file and I am calling
scriptEngine.Evaluate(FileScript)
am also setting break point line no to 0. This hits the break point and I am successfully able to debug step by step till line 4

I have also provided option to load script dynamically similar to shown here
engine.Evaluate(File.ReadAllText(path))
Dynamically loaded script executes successfully but the script in this file does not hit break point. I am trying to debug the dynamically loaded script also.

Please let me know is this a limitation or am I missing something.

Thank you.

New Post: Debug script

$
0
0
Hello!

A couple of questions:
  • Which script engine are you using?
  • How are you setting a breakpoint in the dynamically loaded script code?
Thanks!
Viewing all 2297 articles
Browse latest View live




Latest Images